DOCUMENT RESUME 



ED 412 428 



CE 075 031 



TITLE 

INSTITUTION 



SPONS AGENCY 
PUB DATE 
NOTE 

AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Extending the Ladder: From CASAS to Work Keys Assessments. 
Final Report . 

American Coll. Testing Program, Iowa City, lA. ; 
Comprehensive Adult Student Assessment System, San Diego, 
CA. 

National Inst, for Literacy, Washington, DC. 

1997-00-00 

98p. 

CASAS Customer Service, 8910 Clairemont Mesa Blvd. , San 
Diego, CA 92123-1104 . 

Reports - Research ( 143 ) 

MF01/PC04 Plus Postage. 

Adult Basic Education; *Basic Skills; Education Work 
Relationship; Evaluation Methods; *Job Skills; Mathematics 
Tests; On the Job Training; Reading Tests; *Test Use; 
♦Workplace Literacy 

♦Comprehensive Adult Student Assessment System; ♦Work Keys 
(ACT) 



ABSTRACT 



CASAS (Comprehensive Adult Student Assessment System) and 
ACT (American College Testing) have separately developed systems to identify 
the skills needed in the workplace. A research project studied the 
relationship between CASAS* Workforce Learning Systems (WLS) and ACT'S Work 
Keys to determine how the systems could be linked to support individuals as 
they move up the employability skills ladder. The project first conducted a 
content and cognitive review to verify three assumptions: workplace skills 
can be viewed as a continuous and progressive process; the two systems 
provide high quality assessment tools; and there is a link between the 
systems. An empirical study addressed two research questions: whether the two 
assessments in reading and mathematics are sufficiently correlated to permit 
meaningful articulation, and whether the two assessments in reading and 
mathematics are different enough in the range of skill levels covered so that 
articulation will extend the total range of skills covered in individual 
assessments . Study participants took either the WLS (n=383) or the Work Keys 
(n=367) reading assessment and either the WLS (n=379) or the Work Keys 
(n=347) mathematics assessment. The final matched sample contained 494 
participants. Results showed that the two systems are sufficiently correlated 
and sufficiently different to make an integration feasible and useful . It was 
recommended that educators, trainers, and human resource managers use the 
systems in tandem for developing workplace learning programs and placing 
individual learners. (The report contains 16 references. Appendices include a 
list of data collection sites, proposal letter, sample assessment items, 
examinees with usable scores by state and site, and relationship between 
participants and typical examinees.) (KC) 
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Executive Summary 



Overview 



Life-long learning is the watchword of the day. All across America people are increasingly 
aware that staying productive and competitive in the workplace requires staying current, and 
staying current requires continuous investments in skill development. No longer is this the con- 
cern only of those in the education and training sectors, employers and employees are reaching 
this conclusion as well. 

Extending the Ladder focuses on the nation's investments in skill development. It shows how 
assessment systems from two of our country's premier testing organizations — the Comprehen- 
sive Adult Student Assessment System (CASAS) and ACT — can be linked to guide individuals' 
education and training as they progress from basic literacy skills to the level of advanced skills 
required to succeed in an increasingly complex economy. 

CASAS' Workforce Learning Systems provides high quality, valuable assessments of the basic 
skills needed in the workplace. ACT's Work Keys system provides high quality, valuable assess- 
ments of more advanced skills required for employees to be productive in the workplace. Given 
these different levels, these two assessment systems complement each other to form a compre- 
hensive assessment system. 

CASAS and the Workforce Learning Systems 

CASAS began developing its Workforce Learning Systems in the mid 1980s to promote work- 
place learning and workforce preparation. The tools included in the Workforce Learning Sys- 
tems are designed for an employment context, targeting both those currently employed and 
those pursuing employment (the Employability Competency System series). Workforce Learning 
Systems is designed to assess and to meet the training needs of today's diverse workforce, in- 
cluding native and non-native English speakers. Over 12 years of continuing research and de- 
velopment have ensured the system is relevant to today's job and skill requirements.' 



' For more information on CASAS, the Workforce Learning Systems, and the Employability Competency System, 
contact CASAS, 8910Clairemont Mesa Boulevard, San Diego, CA921 23-1 104; telephone (61 9)292-2900 or (800)255- 
1 036; fax (61 9)292-291 0; or visit the CASAS home page on the Internet at http;//www.casas.org. 
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Workforce Learning Systems tools include four key components: 

• A Workplace Analysis that identifies the basic skills and skill levels required by employees 
at a work site for continuous quality improvement. 

• Workplace Appraisals that determine individuals' actual skill levels, and provide pertinent 
information required for establishing training standards and goals. 

• An Instructional Materials Guide that helps trainers locate the most appropriate curricula 
for developing and delivering work site training. The guide correlates over 1 ,000 
commercially available instructional textbooks, videos, and software programs to more 
than 300 competency statements (or learning objectives). 

• Standardized, including Performance-Based, Assessments that measure learners' progress 
and certify their skill attainment, evaluate program results, and provide a reporting system. 



Employability Competency System Appraisal 

The Employability Competency System (ECS), a part of the Workforce Learning Systems, focuses 
on workplace skills, but places a heavier emphasis on skills for access/ngthe labor market than 
do other parts of the Workforce Learning Systems. The ECS includes appraisals and diagnostic 
tests to assess individuals' employability competencies and shape appropriate training strate- 
gies. 

ECS appraisals predict individuals' general reading and mathematics functional levels, while 
the more targeted Reading and Mathematics for Employability pre-tests provide the diagnostic 
information needed to design and deliver appropriate training. ECS pre-employment and work 
maturity checklists provide standardized forms for rating individuals' career awareness and ability 
to interview and fill out job applications. A reporting system provides useful information to 
learners and program staff. 

CASAS assessment instruments measure a wide range of skill levels, providing accurate assess- 
ments for individuals with special learning needs on up to those with high school completion 
level skills. CASAS assessments are scored from below 1 50 to above 250, and classified along a 
five-level scale ranging from Levels A through E. Clear and easily understood "competency 
descriptors" provide information detailing the skills of individuals scoring within each of the 
five levels. The scaled scores, score levels, and competency descriptors are contained in Table 1 . 

CASAS chose to use the ECS reading and mathematics appraisals, form 1 30, for this study be- 
cause it assesses individuals' strengths and weaknesses in relation to the basic skills necessary to 
obtain and to retain a job. 
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Table 1 

CASAS Basic Skills Levels 
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Advanced Adult Secondary 

With some assistance, persons at this level are able to interpret technical information, 
more complex manuals, and material safety data sheets (MSDS). Can comprehend 
some college textbooks and apprenticeship manuals. 


D 


Adult Secondary 

Can read and follow multi-step directions; read and interpret common legal forms 
and manuals; use math in business, such as calculating discounts; create and use 
tables and graphs; communicate personal opinion in written form; write an accident 
or incident report. Can integrate information from multiple texts, charts, and graphs 
as well as evaluate and organize information. Can perform tasks that involve oral 
and written instructions in both familiar and unfamiliar situations. 


C 


Advanced Basic Skills 

Can handle most routine reading, writing, and computational tasks related to their 
life roles. Can interpret routine charts, graphs, and labels; read and interpret a 
simple handbook for employees; interpret a payroll stub; complete an order form 
and do calculations; compute tips; reconcile a bank statement; fill out medical 
information forms and job applications. Can follow multi-step diagrams and written 
instructions; maintain a family budget; and write a simple accident or incident 
report. Can handle jobs and job training situations that involve following oral and 
simple written instructions and diagrams. Persons at the upper end of this score 
range are able to begin GED preparation. 


B 


Intermediate Basic Skills 

Can handle basic reading, writing, and computational tasks related to their life 
roles. Can read and interpret simplified and some authentic materials on familiar 
topics. Can interpret simple charts, graphs, and labels; interpret a basic payroll 
stub; follow basic written instructions and diagrams. Can complete a simple order 
form and do calculations; fill out basic medical information forms and basic job 
applications; follow basic oral and written instructions and diagrams. Can handle 
jobs and/or job training that involve following basic oral or written instructions and 
diagrams if they can be clarified orally. 




Beginning Basic Skills 

Can fill out simple forms requiring basic personal information, write a simple list or 
telephone message, calculate a single simple operation when numbers are given, 
and make simple change. Can read and interpret simple sentences on familiar topics. 
Can read and interpret simple directions, signs, maps, and simple menus. Can handle 
entry level jobs that involve some simple written communication. 


A 


Pre-Literacy 

Very limited ability to read or write. Persons at the upper end of this score range 
can read and write numbers and letters and simple words and phrases related to 
immediate needs. Can provide very basic personal identification in written form 
such as on job applications. Can handle routine entry level jobs that require only 
basic written communication. 
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ACT'S Work Keys system was developed to remedy crucial basic skill deficiencies in the nation's 
current and future workforce. Work Keys helps employers identify skill levels required by their 
jobs, select qualified applicants, and communicate skill requirements to future employees and 
their instructors. Work Keys job profiling supports employers' use of Work Key scores for job 
selection by addressing Equal Employment Opportunity Commission (EEOC) content validation 
requirements. By linking required skill levels to hiring decisions, employers make their "admis- 
sion standards" public. Work Keys also calculates occupational profiles — summaries of job 
profiles across different companies — which help guide students and educators to develop 
those skills and skill levels.^ 

Work Keys consists of four components: 

• Assessments that measure learners' workplace skills, 

• A job profiling system that determines the skills required for competent performance in 
specific jobs, 

• Instructional support that helps instructors teach necessary skills, and 

• A research and reporting system that provides timely and useful information to Work Keys 
participants. 



Work Keys Assessments 



Work Keys currently supports the skill areas of Applied Mathematics, Listening, Writing, Read- 
ing for Information, Applied Technology, Locating Information, Teamwork, and Observation. 
For each skill area, a common skill scale provides the links between job profiling, assessments, 
and instructional support. 

The Work Keys assessments were developed under the guidance of panels comprised of educa- 
tors and employers. The lowest level of a skill is identified as the simplest level for which a 
business or industry would be interested in administering an assessment for hiring purposes. 
Typically that level is well above "none" of the skill, and for this reason, that level has generally 
been given the name "Level 3." 

The highest skill level is identified as the most complex level at which an employee might be 
expected to function without specialized training. Once the top and bottom levels are described, 
the panel determines about how many levels can be distinguished from the bottom to the top of 
the scale, and these levels are named by counting up from the lowest (level 3, level 4, level 5, 
etc.). The skill levels for the reading and mathematics assessments are described in Tables 2 and 
3. These measure examinees on a scale from 3 to 7. 



^ For more information on ACT and the Work Keys system, contact ACT, 2201 North Dodge Street, P.O. Box 168, 
Iowa City, Iowa 52243; telephone (800)WORKKEY; fax (31 9)337-1 725; or visit the ACT home page on the Internet at 
http://www.act.org. 
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Questions at Level 3 measure the examiner's skill in reading short, uncomplicated passages which use elementary 
vocabulary. The reading materials include basic company policies, procedures, and announcements. All of the 
information needed to answer the questions is stated clearly in the reading materials, and the questions focus on 
the main points of the passages. At this level, the wording of the questions and answers is similar or identical to 
the wording used in the reading materials. Questions at Level 3 require the examinee to 
•identify uncomplicated key concepts and simple details; 

•recognize the proper placement of a step in a sequence of events, or the proper time to perform a task; 
•identify the meaning of a word that is defined within the passage; 

•identify the meaning of a simple word that is not defined within the passage; and 

•recognize the application of instructions given in the passage to situations that are also described in the passage. 


4 


At Level 4, the reading passages are slightly more complex than those at Level 3. They contain more detail and 
describe procedures which involve a greater number of steps. Some passages describe policies and procedures 
with a variety of factors which must be considered in order to decide on appropriate behavior. The vocabulary, 
while elementary, includes words that are more difficult than those at Level 3. For example, the word "immediately" 
may be used at this level, whereas at Level 3 the phrase "right away" would be used. At this level, the questions 
and answers are paraphrased from the passage. In addition to the skills tested at the preceding level, questions at 
Level 4 require the examinee to 

•Identify important details that are less obvious than those in Level 3; 

•recognize the application of more complex instructions, some of which involve several steps, to describe 
situations; and 

•recognize cause-effect relationships. 


5 


Passages at Level 5 are more detailed, more complicated, and cover broader topics than those at Level 4. Words 
and pnrases may be specialized (e.g., jargon and technical terms), and some words may have multiple meanings. 
Questions at this level typically call for applying information given in the passage to a situation that is not 
specifically described in the passage. All of the information needed to answer the questions is stated clearly in 
the passages, but the examinee may need to take several considerations into account in order to choose the 
correct responses. In addition to the skills tested at the preceding levels, questions at Level 5 require the examinee 
to 

•identify the paraphrased definition of a technical term or jargon that is defined in the passage; 

•recognize the application of technical terms or jargon to stated situations; 

•recognize the definition of an acronym that is defined in the passage; 

•identify the appropriate definition of a word with multiple meanings; 

•recognize the application of instructions from the passage to new situations that are similar to those described 
in the reading materials; and 

•recognize the application of more complex instructions to described situations, including conditionals and 
procedures with multiple steps. 


6 


Passages at Level 6 are significantly more difficult than those at the previous level. The presentation of the 
information is more complex; passages may include excerpts from regulatory and legal documents. The procedures 
and concepts described are more elaborate. Advanced vocabulary, jargon, and technical terms are used. Most 
information needed to answer the questions correctly is not clearly stated in the passages. The questions at this 
level require examinees to generalize beyond the stated situation, to recognize implied details, and to recognize 
the probable rationale behind policies and procedures. In addition to the skills tested at the preceding levels, 
questions at Level 6 require the examinee to 

•recognize the application of jargon or technical terms to new situations; 

• recognize the application of complex instructions to new situations; 

•recognize, from context, the less common meaning of a word with multiple meanings; 

•generalize from the passage to situations' not described in the passage; 

• identify implied details; 

•explain the rationale behind a procedure, policy, or communication; and 
•generalize from the passage to a somewhat similar situation. 


7 


The questions at Level 7 are similar to those at Level 6 in that they require the examinee to generalize beyond the 
stated situation, to recognize implied details, and to recognize the probable rationale behind policies and 
procedures. However, the passages are more difficult; the density of information is higher; the concepts are more 
complex, and the vocabulary is more difficult. Passages include jargon and technical terms whose definitions 
must be derived from context. In addition to the skills tested at the proceeding levels, questions at Level 7 require 
the examinee to 

•recognize the definitions of difficult, uncommon jargon or technical terms, based on the context of the reading 
materials; and 

•figure out the general principles underlying described situations and apply them to situations neither described 
in nor completely similar to those in the passage. 
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Table 3 

Work Keys Applied Mathematics 
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Problems at Level 3 measure the examinee's skill in performing basic mathematical operations (addition, 
subtraction, multiplication, and division) and conversions from one form to another, using whole numbers, 
fractions, decimals, or percentages. Solutions to problems at Level 3 are straightforward, involving a single 
type of mathematical operation. For example, the examinee might be required to add several numbers or to 
calculate the correct change in a simple financial transaction. Problems at this level translate easily from a 
verbal setup to a mathematical equation. All the information needed to solve the problems is provided in 
logical order and no unrelated information is included. Problem setups may include units of measurement. 
However, with the exception of dollars and cents, these units function solely as labels and are not involved 
in actual calculations. 


4 


Problems at Level 4 measure the examinee's skill in performing one or two mathematical operations, such 
as addition, subtraction, or multiplication, on several positive or negative numbers. (Division of negative 
numbers is not covered until Level 5.) Problems may require adding commonly known fractions, decimals, 
or percentages (e.g., 1/2, .75, 25%), or adding three fractions that share a common denominator. At this 
level, the examinee is also required to calculate averages, simple ratios, proportions, and rates, using whole 
numbers and decimals. Problems at this level require the examinee to reorder verbal information before 
performing calculations. The examinee must read the entire problem carefully to determine which operation(s) 
to perform and in what order. For some problems, examinees must read a simple chart or graph to obtain 
the information needed to solve the problem. 


5 


Problems at Level 5 require the examinee to look up and calculate single-step conversions within English or 
non-English systems of measurement (e.g., converting from ounces to pounds or from centimeters to meters) 
or between systems of measurement (e.g., converting from centimeters to inches). These problems also 
require calculations using mixed units (e.g., hours and minutes). Problems at this level contain several steps 
of logic and calculation. The examinee must determine what information, calculations, and unit conversions 
are needed to find a solution. For example, the examinee might be asked to calculate perimeters and areas 
of basic shapes, to calculate percent discounts or markups, or to complete a balance sheet or order form. 


6 


Problems at Level 6 measure the examinee's skill in using negative numbers, fractions, ratios, percentages, 
and mixed numbers in calculations. For example, the examinee might be required to calculate multiple 
rates, to find areas of rectangles or circles and volumes of rectangular solids, or to solve problems that 
compare production rates and pricing schemes. The examinee might need to transpose a formula before 
calculating or to look up and use two formulas in conversions within a system of measurement. Level 6 
problems may also Involve identifying and correcting errors in calculations. Problems at Level 6 may require 
considerable translation from verbal form to mathematical expression. They generally require considerable 
set-up and involve multiple-step calculations or conversions. 


7 


Problems at Level 7 require multiple steps of logic and calculation. For example, the examinee may be 
required to convert between systems of measurement that involve fractions, mixed numbers, decimals, or 
percentages; to calculate multiple areas and volumes of spheres, cylinders, and cones; to set up and 
manipulate complex ratios and proportions; or to determine the better economic value of several alternatives. 
Problems may involve more than one unknown, nonlinear functions, and application of basic statistical 
concepts (e.g., error of measurement). The examinee may be required to locate errors in multiple-step 
calculations. At this level, problem content or format may be unusual, and the information presented may 
be incomplete or implicit, requiring the examinee to derive the information needed to solve the problem 
from the setup. 
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Work Keys used the Reading for Information and Applied Mathematics assessments for this 
study since, of the Work Keys skill areas, these tests are the most closely aligned to the two 
CASAS ECS appraisals used in this study. 



Extension Ladder: Workforce Learning SystemsAVork Keys 

Both CASAS' Workforce Learning Systems and ACT's Work Keys system are highly effective, 
workforce focused assessment systems. Neither, however, is targeted broadly enough to assess 
the extensive range of skill levels existent in America's workforce. 

Separately, Workforce Learning Systems and the Work Keys system provide valuable informa- 
tion about individuals' basic skill levels. Together, they provide an "assessment continuum" that 
can be likened to an extension ladder, with the Workforce Learning Systems assessments form- 
ing the lower ladder rungs, and the Work Keys system assessments extending beyond the 
Workforce Learning Systems' highest skill levels and forming the higher ladder rungs (see Fig- 
ure /). There is a critical need for implementing a workforce focused assessment system that 
addresses the broad continuum of skill levels, from the "first rung of the ladder" (basic skills), up 
to the "top rungs of the ladder" (more advanced skills). 

The Extending the Ladder project began with an assumption that CASAS' Workforce Learning 
Systems, which uses assessments and instructional materials tailored to the current skill levels of 
learners, will help individuals at the lower levels of the skills continuum begin to acquire the 
skills needed for a particular job or occupation. Individuals at the upper levels of the skills 
continuum need higher level assessments. The Extending the Ladder project began with the 
assumption that Work Keys system assessments would be better at measuring the more ad- 
vanced skills these individuals would have. 

The Extending the Ladder project assumed that individuals in the middle of the skills continuum 
might be well served by both assessment systems. Just as an extension ladder needs a solid, 
overlapping section to ensure its stability and structural integrity, the Workforce Learning Sys- 
tems and Work Keys system would benefit from an overlap in skills assessment, in the middle of 
the continuum, to provide a smooth transition from one assessment system to the next. A strong 
correlation over some middle skill range would provide a useful transition from lower level 
basic skills to higher level, more advanced skills. 

This study was designed to determine whether the Workforce Learning Systems and Work Keys 
system assessments can be linked to create an assessment scale extension ladder that can sup- 
port learners throughout the skills acquisition process. If the two systems can be linked in this 
way, they will help learners begin their studies at their current skill levels, and continue their 
studies until they have acquired the skills needed for the jobs they are seeking. An added benefit 
is that workforce and worksite training programs will clearly see a solid link joining the Workforce 
Learning Systems and Work Keys system. 
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Research Plan 



In order to help educators, trainers, employers, and employees better assess the skills individu- 
als need to succeed in the workplace, ACT and CASAS established a plan to study the relation- 
ship between two of their assessment systems — ACT's Work Keys and CASAS' Workforce 
Learning Systems — to determine how the systems could be linked to support individuals as 
they move up the employability skills ladder. The two organizations have cooperated to better 
determine how linkages between the two systems might establish a comprehensive continuum 
of assessment to better support skills development. 

Major Project Themes 



• This research project was a joint cooperative venture between two private organizations, 
CASAS and ACT, to develop a comprehensive assessment model for the nation's workforce. 
This cooperative venture is one of an increasing number of workplace skill development 
initiatives envisioned by and carried out by mainstream private organizations. 

• The effective use of these two systems, functioning in tandem, can identify training needs at 
the skill level of the individual and support continued training to the requisite levels mandated 
by business and industry for occupations requiring higher skill levels. 

• Both systems use job analysis services and assessment services of individual skills to enable 
learners and employers to identify the skills needed for job success as well as identify the 
additional training needed to be productive on the job. Using this information, a trainer is 
better able to guide individuals as they prepare for and maintain successful performance in 
today's competitive workforce. 



Project Assumptions 

• Assessment of basic and advanced workplace skills can be viewed as a continuous and 
progressive process. 

• CASAS' Workforce Learning Systems provides high quality, valuable assessments of the basic 
skills needed in the workplace. 

• act's Work Keys system provides high quality, valuable assessments of more advanced skills 
required for employees in the workplace. 

• CASAS' Workforce Learning Systems and ACT's Work Keys reading assessments measure 
similar content and have some overlap in the range of skill levels they measure. The Workforce 
Learning Systems and Work Keys mathematics assessments also measure similar content and 
have some overlap in the range of skill levels they measure. 

• There is a complementary link between the Workforce Learning Systems and Work Keys 
system. 
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XIX 



Goals and Objectives 



The overarching goal of the Extending the Ladder project was to provide educators, trainers, 
employers and employees with a common language and articulated assessments for communi- 
cating about basic and advanced workplace skills and the standards for measuring them. Spe- 
cific objectives included; 

• Determining the degree to which CASAS' Workforce Learning Systems and ACT's Work Keys 
assessments can be linked to provide a continuous, progressive assessment system from basic 
through advanced workplace skills. 

• Providing guidance on appropriate assessments as learners make the transition through skill 
levels on the Workforce Learning Systems and Work Keys assessment systems. 

• Providing information to support establishing comprehensive assessment systems in a variety 
of workforce settings (e.g. secondary and postsecondary education programs, business and 
industry, workforce development centers, or social service agencies). 



Research Questions 

• Are the Workforce Learning Systems and Work Keys assessments in reading and mathematics 
sufficiently highly correlated in the range of skill levels covered to permit meaningful 
articulation of the two scales? 

• Are the Workforce Learning Systems and Work Keys assessments in reading and mathematics 
different enough in the range of skill levels covered so that articulation of the two scales will 
extend the total range of skills covered in the individual assessments? 



Methodology 

The Extending the Ladder project team f/rsf conducted a content/cognitive review to verify the 
study's assumptions, and then conducted an empirical study to address the study's two research 
questions. 

Content/ Cognitive Review 

The project's content/cognitive review verified the project's five assumptions. In order for the 
Workforce Learning Systems and Work Keys system assessments to be linked, they need to 
measure similar content (reading and mathematics), and they need to contain some items mea- 
suring skills at comparable levels of difficulty (the area of overlap on the extension ladder). The 
content/cognitive review found considerable similarity in the content measured by the two 
systems, with both systems relating assessment items to "real-life" workforce tasks, as opposed 
to academic tasks. The range of difficulty covered by the two systems' assessments also differ, 
and the review found enough overlap to make a link, and enough difference to make linking 
worthwhile. 
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Empirical Study 



Once the assumptions were verified, the project team designed and conducted an empirical 
study to address the project's two research questions. 

For the empirical study, adult learners were asked to take both ECS and Work Keys reading 
assessments, and/or both ECS and Work Keys mathematics assessments. Comparisons were 
then made, for each individual, between scores on the paired tests in order to determine the 
linkage between the ECS and Work Keys system assessments. 

Site Selection. Data were collected from 27 sites across eight states. States were selected for 
participation from across the country. Participating sites were selected from locations with 
workforce literacy programs, to ensure that individuals participating in the assessments would 
represent the adult population in workforce literacy programs. 

Participant Sampling Procedures. Sites were asked to randomly select 20 to 30 learners to 
participate in the study. To encourage participation, individuals were awarded dictionaries for 
taking the reading assessments and calculators for taking the mathematics assessments. 

Instrumentation. Individuals participating in the reading study took both the ECS form 1 30 
reading appraisal and the Work Keys Reading for Information assessment. Those participating in 
the mathematics study took both the ECS form 1 30 mathematics appraisal and the Work Keys 
Applied Mathematics assessment. 

Data Collection. The study used scores from 494 individuals: 193 with only reading test 
scores, 1 63 with only mathematics test scores, and 1 38 with both reading and mathematics 
test scores. 

Delimitations. The sampling method used for this study imposed some limitations on the use of 
the study results. 

• The study results are useful in implementing a comprehensive assessment system including 
both the Workforce Learning Systems and Work Keys system. However, they are only relevant 
for the Workforce Learning System's ECS form 130 reading and mathematics appraisals and 
the Work Keys' Reading for Information and Applied Mathematics assessments, and not for 
any other Workforce Learning Systems or Work Keys assessments. 

• Because the participants in this study came from workplace and workforce development 
programs, recommendations from the study are most relevant for individuals in these types of 
programs. 

• Study participants were only representative of a subset of the workplace and workforce 
development programs served by the Workforce Learning Systems and Work Keys system, 
and may not be representative of the universe of programs served by these two systems. 

• This study focused on reading and mathematics assessments. Both the Workforce Learning 
Systems and Work Keys system have additional system components to assist human resource 
development efforts, but linkages for these other system components were not included in 
this study. 
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• The study participants were adults. Therefore, the results may not be readily applicable to 
programs serving younger individuals. 



Research Results 

The empirical study answered both of the research questions, finding that: 

• The Workforce Learning Systems and Work Keys reading and mathematics assessments are 
sufficiently similarto warrant linking the two systems, and 

• The Workforce Learning Systems and Work Keys reading and mathematics assessments are 
sufficiently different to provide more information when linked than when either system is 
used on its own. 

The relationships between the CASAS Workforce Learning Systems and the ACT Work Keys 
system do offer the anticipated extension ladder effect. The CASAS Workforce Learning Systems 
scales extends below and differentiates more levels below the ACT Work Keys system level 3 in 
both skill areas. The ACT Work Keys system differentiates more levels at the upper end and 
extends above the CASAS Workforce Learning Systems scales. Therefore, the CASAS Workforce 
Learning Systems and ACT Work Keys system can be articulated, with their effectiveness for an 
individual dependent on the level at which that person enters an instructional program and on 
the intended use of the score. 

Although the relationship between the score scales for each skill area is strong enough to permit 
scores on one test to be estimated from scores on the other, scores on one test may not be 
directly substituted for scores on the other for the same skill area. 

Interpreting the Results 

By articulating the Workforce Learning Systems and Work Keys system, it is possible to assess 
individuals upon entering an instructional program at whatever level they are currently func- 
tioning, and to provide assessment to support their eventual transitions into the workplace as 
they reach the skill levels required by employers. 

The next four tables (Tables 4 to 7) may be used to estimate what level a person with Workforce 
Learning Systems scores might achieve on the Work Keys assessments, and what level a person 
with Work Keys scores might achieve on the Workforce Learning Systems assessments. They 
have been designed to help practitioners use the study results to provide learners, employees, 
and others who have scores on one assessment with information about how they might score on 
the other. 

There are two tables for the reading assessments (Tables 4 and 6): one based on the CASAS 
Workforce Learning Systems scale (Table 4) and the other based on the ACT Work Keys scale 
(Table 6). There are also two similar tables for the mathematics assessments (Tables 5 and 7). All 
of the tables are to be read in the same way. Start by looking in the leftmost column for the score 
level that matches the score the person already has. Then, read across that row to find the 
chances in 1 00 (the probability) that the individual would have each of the different level scores 
on the other test for the same skill area. 
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Table 4, for example, shows that if an individual has an ACT Work Keys reading score of level 4, 
he or she has 2 chances in 1 00 of having a CASAS Workforce Learning Systems reading level of 
B, 44 chances in 100 of having a CASAS Workforce Learning Systems reading level of C, 47 
chances in 100 of having a CASAS Workforce Learning Systems reading level of D, and 7 
chances in 1 00 of having a CASAS Workforce Learning Systems level of E. That is, if this person 
were to take the Workforce Learning Systems reading test, he or she would probably score at the 
C or D level. 

These tables make it clear that a level on either test is related to more than one level on the 
other. The information presented in these four tables, along with other information about exam- 
inees, can help educators, training specialists, and counselors make better instructional deci- 
sions. 



Table 4 

Expected CASAS Workforce Learning Systems Reading Levels 
as Indicated by ACT Work Keys Scores 


ACT Work Keys 
Reading for 
Information Score 


Chances in 100 of each CASAS Workforce Learning 
Systems Reading Level 


Level A Level B Level C Level D Level E 


Below 3 (N=53) 
Level 3 (N=55) 
Level 4 {N=^ 28) 
Level 5 (N=58) 
Level 6 (N=26) 
Level 7 (N=11) 


4 43 53 

16 66 14 4 

2 44 47 7 

2 12 41 45 

4 8 8 81 

9 91 



Note: The rows may not sum to 100 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 



Table 5 

Expected CASAS Workforce Learning Systems Mathematics Levels 
as Indicated by ACT Work Keys Scores 


ACT Work Keys 
Applied 

Mathematics Score 


Chances in 100 of each CASAS Workforce Learning 
Systems Mathematics Level 


Level A Level B Level C Level D Level E 


Below3(N=51) 
Level 3 (N=105) 
Level 4 (N =91) 
Level 5 (N=35) 
Level 6 (N=13) 
Level 7 (N=6) 


10 76 14 

55 43 1 1 

26 57 16 

9 46 43 3 

8 54 38 

17 83 



Note: The rows may not sum to 100 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 
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Table 6 

Expected ACT Work Keys Reading Levels as Indicated by CASAS 
Workforce Learning Systems Scores 


CASAS Workforce 
Learning Systems 
Reading Score 


Chances in 1 00 of each ACT Work Keys 
Reading for Information Level 


Below 3 Level 3 Level 4 Level 5 Level 6 Level 7 


Lev^ A (N=2) 
Level B (N=36) 
Level C (N=121) 
Level D (N=39) 
;Level E (N=12) 


100 

64 25 6 3 3 

22 28 44 5 2 

8 63 25 2 i 

3 13 38 31 15 



Note: The rows may not sum to 100 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 



Table 7 

Expected ACT Work Keys Mathematics Levels as Indicated 
by CASAS Workforce Learning Systems Scores 


CASAS Workforce 
Learning Systems 
Mathematics Score 


Chances in 1 00 of each ACT Work Keys 
Applied Mathematics Level 


Below 3 Level 3 Level 4 Level 5 Level 6 Level 7 


Level A (N=5) 
Level B (N=124) 
Level C (N=121) 
Level D (N=39) 
Level E (N=12) 


100 

32 47 19 3 

6 37 43 13 1 

3 38 38 18 3 

8 , 8 42 42 



Note: The rows may not sum to 100 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 



Conclusions 

• Both CASAS' Workforce Learning Systems and ACT's Work Keys system measure basic and 
more advanced skills needed for effective participation in the workforce and workplace. 

• There is enough commonality \n both the content and range of skills covered by the Workforce 
Learning Systems and Work Keys system to determine a meaningful connection between the 
two. 

• There is enough difference in the range of skills covered by the Workforce Learning Systems 
and Work Keys system to warrant joining the two together in a basic skills continuum. This 
continuum provides more useful information over a greater range of skill levels than is avail- 
able from either system alone. 
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• The Workforce Learning Systems covers skills that are more basic, and provides more infor- 
mation on the skills of individuals functioning at the lower end of the skills continuum. 

• The Work Keys system covers skills that are more advanced, and provides more information 
on the skills of individuals functioning at the higher end of the skills continuum. 

• The Workforce Learning Systems and Work Keys system complement each other, and can be 
used together to provide a smooth, progressive, and complete skills continuum. 

• A statistical relationship exists between the CASAS Workforce Learning Systems and the ACT 
Work Keys system scales that can be used to estimate scores on one assessment from scores 
on the other. 

• It is not appropriate to substitute scores on one assessment with those from the other. 



Recommendations 

• Educators, trainers, and human resource managers can use the Workforce Learning Systems 
and the Work Keys system in tandem to help individuals progress along a broad skills con- 
tinuum, from their current skill levels to the skill levels required for successful participation in 
the workforce. 

• Educators, trainers, and human resource managers can use the Workforce Learning Systems 
assessments with individuals whose skills are on the lower end of the skills continuum. 

• Educators, trainers, and human resource managers can use the Work Keys system assess- 
ments with individuals whose skills are on the higher end of the skills continuum. 

• Educators, trainers, and human resource managers can use the assessments of either the 
Workforce Learning Systems, the Work Keys system, or the two systems together with indi- 
viduals whose skills are in the mid range of the skills continuum. 



Potential Uses 

The information on linking CASAS' Workforce Learning Systems and ACT's Work Keys has myriad 
uses for education and training providers. Some of these uses are identified below. More de- 
tailed information on these applications is contained in the full report. 



For Those Using the Workforce Learning Systems 

• Individuals who have already been tested with a Workforce Learning Systems assessment can 
use the tables in this publication to estimatehow they would perform on a Work Keys assess- 
ment, and determine whether it would be worthwhile for them to take that assessment now. 
This would be valuable in cases where the individuals assessed with Workforce Learning 
Systems needed to meet a Work Keys standard, perhaps for a new job or a promotion. 
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• Individuals in CASAS-based education and training programs can use Work Keys Occupa- 
tional Profiles, along with the tables in this publication, to estimate the CASAS scale scores 
they would need in order to meet the reading and mathematics requirements for specific 
occupations. Work Keys has developed profiles on a number of occupations by combining 
the results of occupational job analyses conducted in specific companies. 

• Individuals can also use the results from this study, along with their Workforce Learning 
Systems scores, to gauge their progress toward meeting a Work Keys level. 

• Individuals scoring in CASAS level E can gain further information on their skills and abilities 
by taking the Work Keys assessments. Moving from the Workforce Learning Systems to the 
Work Keys system enables these individuals to take advantage of the upward extension of- 
fered by the Work Keys system. 



For Those Using Work Keys 



• Individuals who have already been tested with a Work Keys assessment can use the tables in 
this publication to estimate how they would perform on a Workforce Learning Systems as- 
sessment, and determine whether it would be worthwhile for them to take that assessment 
now. This would be valuable in cases where the individuals assessed with Work Keys needed 
to meet a Workforce Learning Systems standard. 

• Individuals in Work Keys-based education and training programs can use the CASAS Materi- 
als Guide, along with the tables in this publication, to select appropriate curriculum and 
instruction materials. The CASAS guide correlates over 1,000 commercially available in- 
structional textbooks, videos, and software programs to more than 300 competency state- 
ments (or learning objectives). 

• Individuals in Work Keys-based education and training programs can use CASAS research to 
estimate how long it would take them to move from one Work Keys level to the next. 

• Individuals scoring below Work Keys level 3 can gain further information on their skills and 
abilities by taking the Workforce Learning Systems assessments. Moving from the Work Keys 
system to the Workforce Learning Systems enables these individuals to take advantage of the 
downward extension offered by the Workforce Learning Systems. 



For Those Using neither the Workforce Learning Systems nor Work Keys 



Selecting the most appropriate assessment to use with an individual should depend first on the 
purpose of the assessment. If, for example, the purpose is to compare the individual's score with 
a standard based on a Work Keys job profile, a Work Keys assessment is required. If, on the 
other hand, the purpose of the assessment is to provide the individual with appropriate instruc- 
tion based on the CASAS Instructional Materials Guide, a Workforce Learning Systems assess- 
ment is required. 
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• Individuals likely to score below level 3 on the Work Keys assessments would be better 
served by the Workforce Learning Systems. 

• Individuals likely to score at CASAS level E on the Workforce Learning Systems assessments 
would be better served by the Work Keys system. 

For Those Seeking a Comprehensive Assessment System 

The results of this study can benefit the assessment efforts of education and training programs at 
state and local levels. Together, the Workforce Learning Systems and Work Keys system assess 
the broad range of skills programs are likely to encounter. By using these two systems in tan- 
dem, states and localities can operate a comprehensive, articulated assessment system, rather 
than a potpourri of unrelated assessment tests. 

• The results of this study are particularly useful for states and localities establishing one-stop 
systems. One-stop facilities are designed to serve any individuals needing employment and 
training services, from welfare recipients to dislocated workers. With such a broad mandate, 
one-stop centers need a comprehensive assessment system appropriate to all comers. 

• The results of this study will also help states and localities meet their responsibilities under 
the new Welfare Reform mandate. Many welfare recipients will begin their education and 
training programs at skill levels most appropriately measured by the Workforce Learning 
Systems.^ Many welfare recipients will then need to continue their education and training 
programs (before or after employment) until they reach skill levels best measured by the 
Work Keys system. 



^ Three-fourths of the United States' welfare recipients scored at or below level 2 on the National Adult Literacy 
Survey (NALS) test {Barton and Jenkins, 1995, p. 3). 
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Equipping today's workforce with competitive skills offers greater income 
security for the workforce and increased productivity for businesses. 

Two organizations, ACT and CASAS, have each developed a system to identify 
the skills learners and employees need to thrive in the workforce. These 
systems of skill assessment are related directly through job analysis activities 
enabling learners and employers to identify employees who will succeed on 
the job as well as identify the additional skill training needed to be productive 
on the job. These two systems will help employers, employees, learners and 
trainers to maintain our nation's competitive workforce edge among nations. 

Specifically, ACT and CASAS have researched how their systems can aid 
learners and employers by extending the ladder of preparation and assessment 
for youth and adult learners to the levels of skills today's employers need. 

Use of these two systems in tandem can begin training at the skill level of the 
individual and continue that training to the levels businesses require. ACT and 
CASAS have demonstrated how this ladder of skill development can be 
extended to benefit all. 



This report emphasizes the benefits of such a private venture as initiated by 
these two businesses: a venture offering national economic growth by 
maintaining the competitive edge in the global marketplace. 




Roj/ertyT.^Joyies 
_J¥'esid/nt anal CEO 
Nati$/nal All/ance of Business 



'^EST COPY AVAILABLE 



1201 New York Avenue, NW Suite 700 Washington, DC 20005-6143 
202-289-2888 FAX 202-289-1303 TDD 202-289-2977 E-mail info@nab.com www.nab.com 

ATLANTA BOSTON CHICAGO DALLAS LOS ANGELES 



9Q 



xxix 



Preface 



One of the most critical elements in helping individuals prepare for the educational and workforce 
challenges of the twenty-first century is a comprehensive assessment system that provides accu- 
rate and reliable information about individuals' employability skills. The CASASAA/ork Keys 
Study was undertaken by the Comprehensive Adult Student Assessment System (CASAS) and 
ACT as a joint project to link their respective workforce assessment systems. The goal was to 
form a comprehensive assessment system that can be utilized by a variety of agencies, educa- 
tional institutions, business and industry, and workforce development centers. 

This study brings together two large-scale assessment systems, each of which provides impor- 
tant information about people: the assessment component of CASAS' Workforce Learning Sys- 
tems which is designed to measure basic literacy skills within an employment context, and the 
assessment component of the Work Keys system which is designed to assess employability 
skills. By analyzing data from CASAS' Workforce Learning Systems and Work Keys reading and 
mathematics assessments, CASAS and ACT researchers were able to investigate the relationship 
and provide linkages between the two assessment systems. Given the results of this study it is 
now possible to offer individuals, educators, and business and industry a comprehensive as- 
sessment system designed to support individuals' skill development from the most basic to highly 
advanced levels. 

This study addresses the assessment needs of many target audiences including: 

• business and industry trainers and developers of training systems, 

• members of the American Society for Training and Development (ASTD), 

• secondary and postsecondary educators, 

• governmental agencies and organizations involved in workforce development 
and preparation, 

• policy and decision makers, 

• other individuals and agencies involved in assessment and training endeavors. 

This landmark study answers an urgent need for linking two nationally recognized assessment 
systems: CASAS' Workforce Learning Systems and ACT's Work Keys. This study benefits the 
greater society by providing a comprehensive assessment system that will help individuals gain 
the necessary skills to function effectively in the twenty-first century. 

John Hartwig Patricia Rickard Joyce R. McLarty 

CASASAA/ork Keys CASAS Director, Work Keys 

Project Director Executive Director Development 



Joel D. West 

Former Executive Director 
ACT Center for 
Education and Work 
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Overview 



Chapter One 



Life-long learning is the watchword of the day. All across America people are increasingly 
aware that staying productive and competitive in the workplace requires staying current, and 
staying current requires continuous investments in skill development. No longer is this the con- 
cern only of those in the education and training sectors, employers and employees are reaching 
this conclusion as well. 

Extending the Ladder iocuses on the nation's investments in skill development. It shows how 
assessment systems from two of our country's premier testing organizations — the Comprehen- 
sive Adult Student Assessment System (CASAS) and ACT — can be linked to guide individuals' 
education and training as they progress from basic literacy skills to the level of advanced skills 
required to succeed in an increasingly complex economy. 

The Comprehensive Adult Student Assessment System and ACT have long histories in the de- 
sign and delivery of skill assessments. More recently, CASAS, with its Workforce Learning Sys- 
tems and ACT, with its Work Keys system, have developed assessment instruments specifically 
focused on the skills employers indicate employees must have. The Workforce Learning Sys- 
tems and Work Keys also include tools to assess the job skills needed in the workplace, so that 
focused training programs can be developed to bridge the gap between what individuals know 
and can do, and what they need to be able to do to succeed on the job. 

CASAS' Workforce Learning Systems provides high quality, valuable assessments of the basic 
skills needed in the workplace. ACT's Work Keys system provides high quality, valuable assess- 
ments of more advanced skills required for employees to be productive in the workplace. Given 
these different levels, these two assessment systems complement each other to form a compre- 
hensive assessment system. 

How, precisely, are the two systems related? At what range of points do the Workforce Learning 
Systems and Work Keys system assessments overlap? How, finally, can a ladder be extended to 
help individuals beginning their skill development at levels tested by the Workforce Learning 
Systems progress on a continuum up to skill levels employees need as assessed by Work Keys? 

Target Audience 

Extending the Ladder answers these questions. It can help a broad range of educators and train- 
ers better meet the needs of their customers — both learners and employers. By extending a 
ladder linking Workforce Learning Systems and Work Keys skill levels, the study described in 
this publication provides educators and trainers with a continuous assessment system which not 
only assesses individuals' current abilities, but also provides a focus for their further education 
and training. 
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Extending the Ladder is valuable reading for individuals throughout the education and training 
communities. It has value for secondary educators, postsecondary educators, adult educators, 
employment and training providers, and both education and corporate-based workplace train- 
ers. As more and more states begin to build comprehensive education and training systems and 
establish one-stop career centers. Extending the Ladder can play a unifying role, tying together 
all of a state's workforce development programs — and the training needs of the diverse array of 
individuals they serve. 



Value of Assessment 



Today, workforce development is a central economic and social issue. Considerable attention is 
focused on improving employees' productivity. Clearly, this is to the benefit of business and 
industry since it impacts companies' bottom lines. Employees, too, have an interest in their own 
productivity. The more productive they are, the more employable, and the greater their earnings 
and income security. 

Productive employees are well trained and appropriately skilled. The emphasis here is on well 
trained and appropr/ate/yskilled. Training needs to be carefully focused, planned, and executed 
to bring employees' skills to the levels required on their jobs. This type of workforce develop- 
ment can best be accomplished with the aid of a "comprehensive assessment system." 

A comprehensive assessment system is comprised of four main parts: 

• Job Analysis or Needs Assessment - Determining the skills needed in the current and future 
workplace and in specific jobs. 

• Individual Assessment - Determining the skill levels of current and potential employees. 

• Contextual Training - Providing effective training to bridge the gap between the level of 
individuals' skills and the level required on the job. 

• Ongoing Assessment and Certification - Monitoring progress and determining whether 
skills have been mastered. 

The keystone to this system is assessment: assessment of the competencies required on the job; 
assessment of the competencies individuals possess; and re-assessment to monitor progress and 
document skill attainment. 

It is important to note the contextual nature of the comprehensive assessment system. Job analy- 
ses, by definition, assess skill requirements within the context of the workplace and specific 
jobs. Individual assessments, administered before, during, or at the end of a training program, 
also measure people's skills within an employability context. 
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A workplace context is also critical to the training component. Training must focus on applica- 
tions, and provide clear linkages between what is being taught and the work that individuals 
will perform. This is not to say that training should be focused solely on narrow job tasks. 
Training must provide learners with generalizable skills that are required across multiple work 
settings, while clearly relating the skills being taught to real and relevant work applications. 



Benefit to Individuals 



A comprehensive assessment system facilitates individuals' entry into and advancement within 
the job market by providing assistance in three key areas: 

• Information, 

• Training, and 

• Certification. 

A comprehensive assessment system provides individuals with timely and relevant information 
about the skills needed for the jobs and careers they would like to pursue. It also provides them 
with information about their own skill levels, and the education and training they need to ac- 
quire to be competitive. 

A comprehensive assessment system identifies appropriate training for individuals. It also facili- 
tates development of a "portfolio" of transferable skills they can take from one employer to 
another as job opportunities shift. 

A high school diploma does not communicate whether individuals have the skills required to be 
productive and successful in the labor market. Class content varies from school to school or 
even from class to class within a school, and letter grades are incomparable. A comprehensive 
assessment system provides individuals with "competency-based certifications" that clearly spell 
out the skills they possess. Such certifications provide employers with information about the 
skills they themselves have determined are valuable and relevant, through their participation in 
job analyses. As a result, employers are more likely to find the certifications relevant and valu- 
able, and the certifications will help individuals in their job application efforts. 



Benefit to Employers 



The key value of a comprehensive assessment system to employers is the access it gives them to 
individuals with the levels of skills the employers need in their employees - both current and 
potential. By first determining job skill needs, and then training to them, this system meets 
companies' human resource demands. By communicating their job skill needs to educators, 
employers improve their own applicant pools. 

This match-up of employee skills to employer needs is supported through employers' participa- 
tion in job analyses. Initial job analysis or needs assessment research determines what compe- 
tencies companies require, and enables companies to be clear about the skills and training 
needed for success on the job. 
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With a comprehensive assessment system, training can be customized and packaged to meet 
the needs of a specific business or industry. Learning can take place within an appropriate 
context, while still being generalizable to multiple work situations and transferable to different 
work environments. 

A comprehensive needs assessment system can help business and industry gain perspective on 
the total learning needs of their employees, from basic skills through advanced training. There 
are classic examples of companies having invested heavily in technical training, only to dis- 
cover that the majority of their employees did not have the necessary basic skills to profit from 
that training. Effective training is built on a base of high quality job analyses and individual 
assessments. 



Benefit to Educators 

A comprehensive assessment system is of great benefit to educators, too. For educators to be 
successful, they need to provide individuals with skills valued by employers. When students 
know they are learning skills demanded in the job market, their motivation increases and they 
choose to become active learners. 

A comprehensive assessment system creates a common language that enables employers and 
educators to communicate about skill needs. With the information educators gain from employ- 
ers, they can design curriculum and instruction that is both effective (provides individuals with 
the right skills), and efficient (focuses on the skills that make a difference in the workplace). 



Value of Skill Development 

Assessment is key to skill development, and skill development is key to prosperity: individuals', 
companies', and the nation's. A common language of skills provides the basis for assessment 
and for building skills: it defines skill needs and shapes skill instruction, supplying a road map 
for skill development. Without a common language supported by assessment, training can lack 
focus and direction. With such assessment, it can be both effective and efficient. 

Today, while traditional credentials (diplomas and certifications) might provide initial access to 
jobs, it is the skills people posses that enable them to work effectively and efficiently on the job. 
Applied skills, at appropriate levels of proficiency, are what individuals need. While success 
depends on job-specific information and knowledge, as well as generic skills, job specifics are 
constantly expanding and changing. Generic skills help people stay current; they are the keys to 
long term success, advancement, and income security. 

The value of skill development to individuals is well documented. The National Adult Literacy 
Survey (NALS) found that individuals demonstrating higher levels of literacy were more likely to 
be employed, work more weeks in a year, and earn higher wages than individuals demonstrat- 
ing lower levels of proficiency (/C/rsch, eta!., 1993). For years, the U.S. Department of Labor has 
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published statistics showing that those with education beyond high school earn significantly 
more than those with a high school diploma, while those who leave school before receiving a 
diploma earn significantly less (U.S. Department of Labor, 1995'). 

The value of skill development to companies has been less well documented. There have been 
individual companies that have calculated a return on their own training investments and re- 
gional and sectorial studies estimating the return on investments in training; but until recently, 
there was no major national study. However, in 1995, the National Center on the Educational 
Quality of the Workforce at the University of Pennsylvania was able to quantify training's ben- 
efit to companies, demonstrating that a 10 percent increase in the average education of all 
workers within an establishment (or about a year of schooling) yielded an 8.6 percent increase 
in productivity for all industries (National Center on the Education Quality of the Workforce, 
1995). 

As individuals and companies benefit from skill development, so too does the nation. Mainte- 
nance of our global competitive edge is contingent upon competent workforce skill levels. 



Research Plan 

Both CASAS and ACT are committed to skill development. The two organizations share a com- 
mon goal of helping education and training providers improve the skills of both those entering 
and those currently employed in the workforce. Assessments from both organizations are de- 
signed to determine individuals' current skill levels, and then, more importantly, guide them as 
they work to advance those levels. 

To better achieve their common goal, CASAS and ACT established a plan to study the relation- 
ship between two of their assessment systems — CASAS' Workforce Learning Systems and ACT's 
Work Keys — to determine how the systems could be linked to support individuals as they 
move up the employability skills ladder. Instead of looking at how one assessment system might 
compete with or substitute for the other, the two organizations have cooperated to better deter- 
mine how linkages between the two systems might establish a comprehensive continuum of 
assessment to better support skills development. 



' This publication contains data for the years 1980, 1985, 1990, and 1994. Every year, data is collected through the 
Current Population Survey by the Bureau of the Census, U.S. Department of Commerce, and analyzed by the Bureau 
of Labor Statistics, U.S. Department of Labor. The positive relationship between education and earnings has been a 
consistent finding of these studies. 



Project Themes 



The project team formulated the following theme statements which served as the guiding prin- 
ciples for the research project: 

• This research project was a joint cooperative venture between two private organizations, 
CASAS and ACT, to develop a comprehensive assessment model for the nation's 
workforce. This cooperative venture is one of an increasing number of workplace skill 
development initiatives envisioned by and carried out by mainstream private 
organizations. 

• Each organization has developed its own respective assessment system with unique 
expertise inherent in each system. 

• The effective use of these two systems, functioning in tandem, can identify training needs 
at the skill level of the individual and continue that training to the requisite levels 
mandated by business and industry for occupations requiring higher skill levels. 

• Both systems use job analysis services and assessment of individual skills to enable 
learners and employers to identify employees who will succeed on the job as well as 
identify the additional training needed to be productive on the job. Using this information, 
a trainer is better able to guide individuals as they prepare for and maintain successful 
performance in today's competitive workforce. 

• The results of this study can be utilized to offer individuals, educators, and business and 
industry a comprehensive assessment system designed to support individuals' skill 
development from the most basic to highly advanced levels. 

• The utility of the Extension Ladder Assessment Model is adaptable to a wide variety of 
workforce and career development environments. 

• Assessment of basic and advanced workplace skills can be viewed as a continuous and 
progressive process. 

• There is dignity and personal identification associated with one's occupation at all levels 
of the work-related skills continuum. All occupations require mastery of varying levels of 
skill attainment ranging from basic communication and literacy skills through advanced 
technical skills. 



Project Assumptions 

The research for Extending the Ladder was based on five key assumptions: 

• Assessment of basic and advanced workplace skills can be viewed as a continuous and 
progressive process. 
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• CASAS' Workforce Learning Systems provides high quality, valuable assessments of the basic 
skills needed in the workplace. 

• ACT'S Work Keys system provides high quality, valuable assessments of more advanced skills 
required for employees to be productive in the workplace. 

• CASAS' Workforce Learning Systems and ACT's Work Keys reading assessments measure 
similar content and have some overlap in the range of skill levels they measure. The Workforce 
Learning Systems and Work Keys mathematics assessments also measure similar content and 
have some overlap in the range of skill levels they measure. 

• There is a complementary link between the Workforce Learning Systems and Work Keys 
system. 



Goals and Objectives 

The overarching goal of the Extending the Ladder project was to provide educators, trainers, 
employers and employees with a common language and articulated assessments for communi- 
cating about basic and advanced workplace skills and the standards for measuring them. Spe- 
cific objectives included: 

• Determining the degree to which CASAS' Workforce Learning Systems and ACT's Work Keys 
assessments can be linked to provide a continuous, progressive assessment system from basic 
through advanced workplace skills. 

• Providing guidance on appropriate assessments as learners make the transition through skill 
levels on the Workforce Learning Systems or Work Keys assessment systems. 

• Providing information to support establishing comprehensive assessment systems in a variety 
of workforce settings (e.g. secondary and postsecondary education programs, business and 
industry, workforce development centers, or social service agencies). 



Research Questions 

Given these assumptions, the project team for Extending the Ladder developed two foci for this 

study as formulated in the following research questions: 

• Are the Workforce Learning Systems and Work Keys assessments in reading and mathematics 
sufficiently highly correlated in the range of skill levels covered to permit meaningful 
articulation of the two scales? 

• Are the Workforce Learning Systems and Work Keys assessments in reading and mathematics 
different enough in the range of skill levels covered so that articulation of the two scales will 
extend beyond the total range of skills covered in the individual assessments? 



ERIC 




7 



Project Phases 



To meet its goals and objectives, the Extending the Ladder team identified five, slightly overlap- 
ping phases for its research effort: 



Phase 


Task 


Time Frame 


1 


Project Definition 


04/01/96 - 08/30/96 


II 


Data Collection 


09/04/96- 10/30/96 


III 


Data Analysis 


11/01/96-03/01/97 


IV 


Report Writing 


01/01/97-08/30/97 


V 


Dissemination 


09/1 7/97 - 1 2/30/97 
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Chapter Two 



Methodology and 
Instrumentation - 



Both CASAS' Workforce Learning Systems and ACT's Work Keys system are comprehensive 
assessment systems. Both include job analyses and individual assessments in an employability 
context, and both facilitate contextual training. 

CASAS' Workforce Learning Systems was developed to fill a critical void in the employability 
skills measurement and instruction of youth and adult students enrolled in employment 
preparation and workplace training programs. The Workforce Learning Systems provides a 
structure in which learners' strengths and weaknesses are assessed in relation to the basic skills 
necessary to obtain, retain, and advance in a job. This assessment is supported by a workplace 
analysis that determines the skills needed in the workplace, and a curriculum management 
system that links the skills needed in the workplace to instruction. 

act's Work Keys system was developed to remedy crucial basic skill deficiencies in the nation's 
current and future workforce. Work Keys helps employers identify skill levels required by their 
jobs, select qualified applicants, and communicate skill requirements to future employees and 
their instructors. Work Keys job profiling supports employers' use of Work Keys scores for job 
selection by addressing Equal Employment Opportunity Commission (EEOC) content validation 
requirements. By linking required skill levels to hiring decisions, employers make their "admission 
standards" public. Work Keys also calculates occupational profiles — summaries of job profiles 
across different companies — which help guide students and educators to develop those skills 
and skill levels. 

CASAS and the Workforce Learning Systems 

CASAS began developing its Workforce Learning Systems in the mid 1 980s to promote workplace 
learning and workforce preparation. The tools included in the Workforce Learning Systems are 
designed for an employment context, targeting both those currently employed and those pursuing 
employment (the Employability Competency System series). Workforce Learning Systems is 
designed to assess and to meet the training needs of today's diverse workforce, including native 
and non-native English speakers. Over 1 2 years of continuing research and development have 
ensured the system is relevant to today's job and skill requirements.^ 



^ For more information on CASAS, the Workforce Learning Systems, and the Employability Competency System, 
contact CASAS, 891 0 Clairemont Mesa Boulevard, San Diego, CA 921 23-1 1 04; telephone (61 9)292-2900 or (800)255- 
1 036; fax (61 9)292-291 0; or visit the CASAS home page on the Internet at http://www.casas.org. 
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Workforce Learning Systems tools address all four components of a comprehensive assessment 
system: 

• A Workplace Analysis that identifies the basic skills and skill levels required by employees 
at a work site for continuous quality improvement. 

• Workplace Appraisals that determine individuals' actual skill levels, and provide pertinent 
information required for establishing training standards and goals. 

• An Instructional Materials Guide that helps trainers locate the most appropriate curricula 
for developing and delivering work site training. The guide correlates over 1 ,000 commercially 
available instructional textbooks, videos, and software programs to more than 300 
competency statements (or learning objectives). 

• Standardized, including Performance-Based, Assessments that measure learners' progress 
and certify their skill attainment, evaluate program results, and provide a reporting system. 

The Employability Competency System (ECS), a partofthe Workforce Learning Systems, focuses 
on workplace skills, but places a heavier emphasis on skills for access/ng the labor market than 
do other parts of the Workforce Learning Systems. The ECS includes appraisals and diagnostic 
tests to assess individuals' employability competencies and shape appropriate training strategies. 
ECS appraisals predict individuals' general reading and mathematics functional levels, while 
the more targeted Reading and Mathematics for Employability pre-tests provide the diagnostic 
information needed to design and deliver appropriate training. ECS pre-employment and work 
maturity checklists provide standardized forms for rating individuals' career awareness and ability 
to interview and fill out job applications. A reporting system provides useful information to 
learners and program staff. 

CASAS' Workforce Learning Systems stresses the importance of job analyses for effectively 
targeting workforce and workplace training efforts. The first component, the Workplace Analysis 
process, analyzes the reading, mathematics, writing, communication, and critical thinking skills 
and skill levels required for specific jobs. This basic skills blueprint ensures that training efforts 
meet the specific skill demands of specific jobs in American businesses. Employment-related 
skills are assessed through multiple measures, including standardized assessment tools, as well 
as such customized curriculum assessment strategies as logs, projects, and portfolios. The ECS 
pre-employment and work maturity checklists are just some of the standardized, performance- 
based assessments that CASAS has developed for the Workforce Learning Systems and ECS. 

CASAS systems have been validated by the Program Effectiveness Panel of the U.S. Department 
of Education, which determined that learners in educational programs that have adopted key 
CASAS elements 1) demonstrate significant learning gains, 2) demonstrate increased hours of 
participation, and 3) achieve increased goal attainment compared to programs that have not 
adopted the key elements. 
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The CASAS National Consortium, comprising 17 states, has participated in the research and 
development of the Workforce Learning Systems over a 1 2 year period. Consortium members 
from these states represent state employment and workplace training programs, community 
college systems, state adult education agencies, high schools, literacy programs, community- 
based organizations, and correctional institutions. 

Consortium members identify key priority needs for development, help field test and evaluate 
assessment and curriculum management components and processes, assist in providing the 
training needed for implementation, and share successful strategies and outcomes in their states. 
The breadth of the National Consortium ensures that the system and its components are relevant 
for diverse populations and workforce programs across the United States. 

Development of the Workforce Learning Systems 

CASAS' Workforce Learning Systems is specialized, and focused on workplace related 
competencies, and skills linked to appropriate curriculum, assessment, and evaluation systems. 
In developing the Workforce Learning Systems, CASAS surveyed business and industry 
representatives and employment and training professionals, and reviewed existing company 
and employment and training program curricula to determine the competencies required for 
success in the workplace. All Workforce Learning Systems tools — workplace analyses, appraisals, 
materials guides, and assessments — were then focused on these competencies. 

The competencies are highly correlated with the SCANS competencies and foundation skills. 
These competencies and skills, defined by the U.S. Department of Labor's Secretary's Commission 
on Achieving Necessary Skills (SCANS), represent the nation's best thinking about the skills and 
abilities all individuals need in order to function successfully in the modern workplace and 
society.^ 

CASAS multiple choice assessments are created from an Item Bank of more than 5,000 statistically 
reliable and externally validated test questions. These, in turn, are linked to specific competency 
statements within nine content areas: basic communication, consumer economics, community 
resources, health, employment, government and law, computation, learning to learn, and 
independent living. The assessments developed for the Workforce Learning Systems measure 
the high priority competencies relevant to the workforce. 

Workforce Learning Systems is used by companies implementing workplace learning programs. 
Job Training Partnership Act (JTPA) and welfare employment training programs, vocational and 
occupational education providers, community colleges, regional occupational programs and 
centers, workplace literacy providers, unified school districts, community-based organizations, 
and correctional institutions. 



^ See CASAS Competencies 1 996 (San Diego, CA: CASAS, 1996) for a correlation of CASAS competencies to SCANS 
competencies, and a correlation of SCANS competencies to CASAS competencies. 
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Employability Competency System Appraisal 



CASAS chose to use the ECS reading and mathematics appraisals, form 1 30, for this study be- 
cause it assesses individuals' strengths and weaknesses in relation to the basic skills necessary to 
obtain and to retain a job. Each appraisal takes 25 minutes to administer and measures partici- 
pants' performance on reading and mathematics applications encountered in the workplace. 
The ECS form 130 appraisal was chosen over other Workforce Learning Systems appraisals 
because it accurately assesses up to a higher level of skills, and is likely to provide a more 
significant link with ACT's Work Keys assessments. 

Reading. The reading section of the ECS appraisal tests participants' reading skills within an 
employability context. The appraisal includes items requiring participants to read and answer 
questions about 1 ) a job ad in a newspaper, 2) an employee's personnel record, 3) a job appli- 
cation letter, 4) a medical history report, 5) an employee handbook, 6) a job description, or 7) 
newspaper articles covering, for example, a labor contract, the importance of a high school 
diploma, or occupational staffing patterns. It also includes items requiring participants to inter- 
pret and answer questions about graphs, which might, for example, chart work activity in a 
business over time. 

Mathematics. The mathematics section of the ECS appraisal also tests mathematics skills within 
an employability context. It includes items requiring participants to interpret and make calcula- 
tions from 1 ) pay stubs, 2) pie charts of budget figures, 3) time cards, or 4) bar graphs of poverty 
levels, as well as word problems requiring participants to compute real heights from a scale 
drawing, a perimeter, the average number of customers visiting a store daily, the cost of carpet- 
ing a living room, or the number of different outfits that can be put together from a specified 
wardrobe. This appraisal also includes basic addition, subtraction, multiplication, and division 
problems. 

Basic Skills Levels. CASAS assessment instruments measure a wide range of skill levels, provid- 
ing accurate assessments for individuals with special learning needs on up to those with high 
school completion level skills. CASAS assessments are scored from below 150 to above 250, 
and classified along a five-level scale ranging from Levels A through E. Clear and easily under- 
stood "competency descriptors" provide information detailing the skills of individuals scoring 
within each of the five levels. The scaled scores, score levels, and competency descriptors are 
contained in Table 2.1 . (See Appendix C for sample assessment items in reading and mathemat- 
ics for CASAS Levels A and E.) 
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Scaled 

Scores 



290 




245 



240 



236 

230 



225 



220 



215 



210 



206 



200 



190 



180 

190 



t 



Table 2.1 

CASAS Basic Skills Levels 


JSeASAS^ 
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E 


Advanced Adult Secondary 

With some assistance, persons at this level are able to interpret technical Information, 
more complex manuals, and material safety data sheets (MSDS). Can comprehend 
some college textbooks and apprenticeship manuals. 


D 


Adult Secondary 

Can read and follow multi-step directions; read and Interpret common legal forms 
and manuals; use math in business, such as calculating discounts; create and use 
tables and graphs; communicate personal opinion in written form; write an accident 
or Incident report. Can integrate information from multiple texts, charts, and graphs 
as well as evaluate and organize information. Can perform tasks that involve oral 
and written instructions In both familiar and unfamiliar situations. 


C 


Advanced Basic Skills 

Can handle most routine reading, writing, and computational tasks related to their 
life roles. Can interpret routine charts, graphs, and labels; read and interpret a 
simple handbook for employees; interpret a payroll stub; complete an order form 
and do calculations; compute tips; reconcile a bank statement; fill out medical 
information forms and job applications. Can follow multi-step diagrams and written 
instructions; maintain a family budget; and write a simple accident or incident 
report. Can handle jobs and job training situations that involve following oral and 
simple written instructions and diagrams. Persons at the upper end of this score 
range are able to begin GED preparation. 


B 


Intermediate Basic Skills 

Can handle basic reading, writing, and computational tasks related to their life 
roles. Can read and interpret simplified and some authentic materials on familiar 
topics. Can interpret simple charts, graphs, and labels; interpret a basic payroll 
stub; follow basic written instructions and diagrams. Can complete a simple order 
form and do calculations; fill out basic medical information forms and basic job 
applications; follow basic oral and written instructions and diagrams. Can handle 
jobs and/or job training that involve following basic oral or written instructions and 
diagrams if they can be clarified orally. 




Beginning Basic Skills 

Can fill out simple forms requiring basic personal information, write a simple list or 
telephone message, calculate a single simple operation when numbers are given, 
and make simple change. Can read and interpret simple sentences on familiar topics. 
Can read and interpret simple directions, signs, maps, and simple menus. Can handle 
entry level jobs that involve some simple written communication. 


A 


Pre-Literacy 

Very limited ability to read or write. Persons at the upper end of this score range 
can read and write numbers and letters and simple words and phrases related to 
immediate needs. Can provide very basic personal identification in written form 
such as on job applications. Can handle routine entry level jobs that require only 
basic written communication. 



CASAS, 1997 
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ACT and Work Keys 

A comprehensive program for assessing and developing workplace skills, the Work Keys system 
was developed by ACT in the early 1 990s in partnership with the charter states of Illinois, Iowa, 
Michigan, Ohio, Oregon, and Tennessee, the Community College System of California, numer- 
ous other educational organizations, and a cross-section of American business. The Work Keys 
system is a coordinated set of programs and services designed to help individuals, employers, 
and educators work together to improve the skills of the workforce.'' 

Work Keys skill scales provide a common language for describing the levels of foundational 
skills required by jobs as well as those demonstrated by individuals. Comparison of the skill 
levels required by a job (the job profile) with the skill levels demonstrated by an individual 
supports identification of any skill deficiencies, which can then be addressed through education 
and training. Applicants who meet the job profile requirements possess the foundational skills 
needed to perform competently on that job. These skills, together with any needed job-specific 
skills, make them prime candidates for employment. 

Work Keys consists of four components: 

• Assessments that measure learners' workplace skills, 

• A job profiling system that determines the skills required for competent performance in spe- 
cific jobs, 

• Instructional support that helps instructors teach necessary skills, and 

• A research and reporting system that provides timely and useful information to Work Keys 
participants. 

Work Keys currently supports the skill areas of Applied Mathematics, Listening, Writing, Read- 
ing for Information, Applied Technology, Locating Information, Teamwork, and Observation. 
For each skill area, a common skill scale provides the links between job profiling, assessments, 
and instructional support. 



Development of the Work Keys System and Skill Levels 



The comparability of Work Keys assessment scores and job profile levels is made possible by a 
highly professional and specialized development process that is unique to the ACT Work Keys 
system. Development of each Work Keys assessment begins with the development of the skill 
scale. A panel of educators and employers meet with ACT staff to carefully circumscribe the 
skill itself and to describe a series of levels within that skill. 



'' For more information on ACT and the Work Keys system, contact ACT, 2201 North Dodge Street, P.O. Box 168, 
Iowa City, Iowa 52243; telephone (800)WORKKEY; fax (31 9)337-1 725; or visit the ACT home page on the Internet 
at http://www.act.org. 



The lowest level of the skill is identified as the simplest level for which a business would be 
interested in administering an assessment for hiring purposes. Typically that level is well above 
"none" of the skill, and for that reason, for multiple choice tests, that level has generally been 
given the name "Level 3." 

The highest level is identified as the most complex level at which an employee might be ex- 
pected to function without specialized training. Once the top and bottom levels are described, 
the panel determines about how many levels can be distinguished from the bottom to the top of 
the scale, and these levels are named by counting up from the lowest (Level 3, Level 4, Level 5, 
etc.). Note that since Listening and Writing scores are based on scoring the examinee's actual 
response, scores for those skill areas go from 0 (no usable information written) to 5 (perfect 
response). 

Once the skill scale has been described, ACT staff develop the job analysis (job profiling) and 
assessment components of the system. The Work Keys system job analysis (job profiling) com- 
ponent consists of a training process to guide qualified individuals to learn the Work Keys job 
analysis system, an authorization system to permit these trained individuals to develop job 
profiles for employers and other clients, and a software system to support them as they perform 
this function. 

Job profiling identifies the Work Keys skills and skill levels needed to competently perform a 
specific job in a particular company. A set of materials that comprise both benchmark defini- 
tions and examples for each level of skill are developed for each skill area. These are developed 
and tried out by Work Keys staff industrial/organizational psychologists to ensure that incum- 
bent workers are able to understand and provide sound information about the skill level re- 
quired by their jobs. Once staff are able to profile the job successfully for the skill area, the 
materials are incorporated into the training program for Work Keys' authorized job profilers, 
enabling them to also profile jobs for that skill area. 

Job profiles are designed to be used by companies to select new employees or identify training 
needs among current employees, as well as to convey expectations for employee skill levels to 
educators. Job profiles are used by individual businesses to address EEOC content validation 
standards and are proprietary to the company that developed them. ACT does not release indi- 
vidually identifiable job profiles to others. 

However, in order to meet the information needs of educators more fully, ACT has developed 
occupational profiles. Occupational profiles are constructed by averaging the data for job pro- 
files for jobs in the Work Keys data base that have the same associated Dictionary of Occupa- 
tional Titles (DOT) code or are part of some other grouping of interest. These occupational 
profiles are made available for guidance and instructional uses since they provide key informa- 
tion about the nature and levels of skills actually required. Because they are not associated with 
a specific job in a specific company, occupational profiles cannot be used as part of selection 
criteria. 
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Work Keys Assessments 



Work Keys used the Reading for Information and Applied Mathematics assessments for this 
study since, of the Work Keys skill areas, these tests are the most closely aligned to the two 
CASAS ECS appraisals used in this study. Each assessment requires 40 minutes administration 
time and measures the examinee on a scale from 3 to 7. These Work Keys assessments do not 
include the interpretations of graphs and charts that are included in the ECS reading and math- 
ematics assessments since these skills are covered in the separate Work Keys Locating Informa- 
tion skill area. 

Reading for Information. The Reading for Information assessment measures an examinee's skill 
in reading and understanding work-related instructions and policies. The reading passages and 
questions in the assessment are based on the actual demands of the workplace. Passages take 
the form of memos, bulletins, notices, letters, policy manuals, and governmental regulations. 
Such materials differ from the expository and narrative texts used in most reading instruction, 
which are usually written to facilitate reading. Workplace communication is not necessarily 
well written or targeted to the appropriate audience. Because the Reading for Information as- 
sessment uses workplace texts, the assessment is more reflective of actual workplace condi- 
tions. Examinees are given 40 minutes to answer 30 multiple-choice questions. 

The reading materials and related multiple-choice questions comprise five levels of complexity, 
with Level 3 being the least complex and Level 7 the most complex. Although Level 3 is the 
least complex, the questions require a level of reading skill well above simple decoding. The 
levels build on each other, each incorporating the skills assessed at the preceding levels. Skill 
levels are described in Table 2.2. 

Work Keys job profiles indicate that Reading for Information is a relevant skill for virtually every 
job, and that the level of skill needed is within the Work Keys scale (i.e., at least level 3 and not 
above level 7). Based on 61 3 jobs profiled, 99.2% were found to be within the scale, with .8% 
below the scale and none above. 
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Table 2.2 

Work Keys Reading for Information 


?fWork , 
TjKeysV 
->~tevel^ 


i 'M- ' Level Description 'i'-' 'l' - 




3 


Questions at Level 3 measure the examiner's skill in reading short, uncomplicated passages which use elementary 
vocabulary. The reading materials include basic company policies, procedures, and announcements. All of the 
information needed to answer the questions is stated clearly in the reading materials, and the questions focus on 
the main points of the passages. At this level, the wording of the questions and answers is similar or identical to 
the wording used in the reading materials. Questions at Level 3 require the examinee to 

• identify uncomplicated key concepts and simple details; 

•recognize the proper placement of a step in a sequence of events, or the proper time to perform a task; 

• identify the meaning of a word that is defined within the passage; 

•identify the meaning of a simple word that is not defined within the passage; and 

•recognize the application of instructions given in the passage to situations that are also described in the passage. 


4 


At Level 4, the reading passages are slightly more complex than those at Level 3. They contain more detail and 
describe procedures which involve a greater number of steps. Some passages describe policies and procedures 
with a variety of factors which must be considered in order to decide on appropriate behavior. The vocabulary, 
while elementary, includes words that are more difficult than those at Level 3. For example, the word "immediately" 
may be used at this level, whereas at Level 3 the phrase "right away" would be used. At this level, the questions 
and answers are paraphrased from the passage. In addition to the skills tested at the preceding level, questions at 
Level 4 require the examinee to 

• identify important details that are less obvious than those in Level 3; 

•recognize the application of more complex instructions, some of which involve several steps, to describe 
situations; and 

•recognize cause-effect relationships. 


5 


Passages at Level 5 are more detailed, more complicated, and cover broader topics than those at Level 4. Words 
and phrases may be specialized (e.g., jargon and technical terms), and some words may have multiple meanings. 
Questions at this level typically call for applying information given in the passage to a situation that is not 
specifically described in the passage. All of the information needed to answer the questions is stated clearly in 
the passages, but the examinee may need to take several considerations into account in order to choose the 
correct responses. In addition to the skills tested at the preceding levels, questions at Level 5 require the examinee 
to 

•identify the paraphrased definition of a technical term or jargon that is defined in the passage; 

•recognize the application of technical terms or jargon to stated situations; 

•recognize the definition of an acronym that is defined in the passage; 

•identify the appropriate definition of a word with multiple meanings; 

•recognize the application of instructions from the passage to new situations that are similar to those described 
' in the reading materials; and 

•recognize the application of more complex instructions to described situations, including conditionals and 
procedures with multiple steps. 


6 


Passages at Level 6 are significantly more difficult than those at the previous level. The presentation of the 
information is more complex; passages may include excerpts from regulatory and legal documents. The procedures 
and concepts described are more elaborate. Advanced vocabulary, jargon, and technical terms are used. Most 
information needed to answer the questions correctly is not clearly stated in the passages. The questions at this 
level require examinees to generalize beyond the stated situation, to recognize implied details, and to recognize 
the probable rationale behind policies and procedures. In addition to the skills tested at the preceding levels, 
questions at Level 6 require the examinee to 

•recognize the application of jargon or technical terms to new situations; 

•recognize the application of complex instructions to new situations; 

•recognize, from context, the less common meaning of a word with multiple meanings; 

•generalize from the passage to situations not described in the passage; 

•identify implied details; 

•explain the rationale behind a procedure, policy, or communication; and 
•generalize from the passage to a somewhat similar situation. 


7 


The questions at Level 7 are similar to those at Level 6 in that they require the examinee to generalize beyond the 
stated situation, to recognize implied details, and to recognize the probable rationale behind policies and 
procedures. However, the passages are more difficult; the density of information is higher; the concepts are more 
complex, and the vocabulary is more difficult. Passages include jargon and technical terms whose definitions 
must be derived from context. In addition to the skills tested at the proceeding levels, questions at Level 7 require 
the examinee to 

•recognize the definitions of difficult, uncommon jargon or technical terms, based on the context of the reading 
materials; and 

• figure out the general principles underlying described situations and apply them to situations neither described 
in nor completely similar to those in the passage. 
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Applied Mathematics The Applied Mathematics assessment measures the examinee's skill in 
applying mathematical reasoning to work-related problems. The test questions require the ex- 
aminee to set up and solve the types of problems and do the types of calculations that actually 
occur in the workplace. This test is designed to be taken with a calculator. As on the job, the 
calculator serves as a tool for problem solving. A formula sheet that includes, but is not limited 
to, all formulas required for the assessment is provided. Examinees are given 40 minutes to 
solve 30 multiple-choice problems. 

This assessment contains questions at five levels of complexity, with Level 3 being the least 
complex and Level 7 being the most complex. The levels build on each other, each incorporat- 
ing the skills assessed at the preceding levels. The skill levels for the assessment are described in 
Table 2.3. 

The Applied Mathematics skill scale and assessment are focused on problem-solving and math- 
ematical reasoning skills, rather than on "higher level" mathematical procedures. Thus, there is 
no formal algebra, geometry, trigonometry, of calculus on the assessment, although knowledge 
of these procedures, and especially of algebra, can be instrumental in allowing an examinee to 
think through and solve the problems. 

The job profiles indicate that Applied Mathematics is a relevant skill for most jobs, and that the 
level of skill needed is within the Work Keys scale (i.e., at least level 3 and not above level 7). 
Based on 604 jobs profiled, 96.5% were found to be within the scale, with 3.3% below the 
scale and .2% above. It is important to recall that Applied Mathematics, like the other Work 
Keys skill scales, is not intended to measure skills that require job-specific training. Thus, many 
jobs that do require use of higher level mathematical procedures (such as trigonometry for 
engineering technicians) were determined in profiling to require both a level within the Work 
Keys Applied Mathematics scale (in the case of engineering technicians, typically a 6 or 7) and 
specialized training in mathematics. (See Appendix C for sample assessment items in reading 
and mathematics for Work Keys Levels 3 and 7.) 
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Work Keys Applied Mathematics 
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Problems at Level 3 measure the examinee's skill in performing basic mathematical operations (addition, 
subtraction, multiplication, and division) and conversions from one form to another, using whole numbers, 
fractions, decimals, or percentages. Solutions to problems at Level 3 are straightforward, involving a single 
type of mathematical operation. For example, the examinee might be required to add several numbers or to 
calculate the correct change in a simple financial transaction. Problems at this level translate easily from a 
verbal setup to a mathematical equation. All the information needed to solve the problems is provided in 
logical order and no unrelated information Is included. Problem setups may Include units of measurement. 
However, with the exception of dollars and cents, these units function solely as labels and are not involved 
in actual calculations. 


4 


Problems at Level 4 measure the examinee's skill in performing one or two mathematical operations, such 
as addition, subtraction, or multiplication, on several positive or negative numbers. (Division of negative 
numbers is not covered until Level 5.) Problems may require adding commonly known fractions, decimals, 
or percentages (e.g., 1/2, .75, 25%), or adding three fractions that share a common denominator. At this 
level, the examinee is also required to calculate averages, simple ratios, proportions, and rates, using whole 
numbers and decimals. Problems at this level require the examinee to reorder verbal Information before 
performing calculations. The examinee must read the entire problem carefully to determine which operation(s) 
to perform and in what order. For some problems, examinees must read a simple chart or graph to obtain 
the information needed to solve the problem. 


5 


Problems at Level 5 require the examinee to look up and calculate single-step conversions within English or 
non-English systems of measurement (e.g., converting from ounces to pounds or from centimeters to meters) 
or between systems of measurement (e.g., converting from centimeters to inches). These problems also 
require calculations using mixed units (e.g., hours and minutes). Problems at this level contain several steps 
of logic and calculation. The examinee must determine what information, calculations, and unit conversions 
are needed to find a solution. For example, the examinee might be asked to calculate perimeters and areas 
of basic shapes, to calculate percent discounts or markups, or to complete a balance sheet or order form. 


6 


Problems at Level 6 measure the examinee's skill in using negative numbers, fractions, ratios, percentages, 
and mixed numbers in calculations. For example, the examinee might be required to calculate multiple 
rates, to find areas of rectangles or circles and volumes of rectangular solids, or to solve problems that 
compare production rates and pricing schemes. The examinee might need to transpose a formula before 
calculating or to look up and use two formulas in conversions within a system of measurement. Level 6 
problems may also involve identifying and correcting errors in calculations. Problems at Level 6 may require 
considerable translation from verbal form to mathematical expression. They generally require considerable 
set-up and involve multiple-step calculations or conversions. 


7 


Problems at Level 7 require multiple steps of logic and calculation. For example, the examinee may be 
required to convert between systems of measurement that involve fractions, mixed numbers, decimals, or 
percentages; to calculate multiple areas and volumes of spheres, cylinders, and cones; to set up and 
manipulate complex ratios and proportions; or to determine the better economic value of several alternatives. 
Problems may involve more than one unknown, nonlinear functions, and application of basic statistical 
concepts (e.g., error of measurement). The examinee may be required to locate errors in multiple-step 
calculations. At this level, problem content or format may be unusual, and the information presented may 
be incomplete or implicit, requiring the examinee to derive the information needed to solve the problem 
from the setup. 
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Extension Ladder: 

Workforce Learning SystemsAVork Keys 



Both CASAS' Workforce Learning Systems and ACT's Work Keys system are highly effective, 
workforce focused assessment systems. Neither, however, is targeted broadly enough to assess 
the extensive range of skill levels existent in America's workforce. Separately, Workforce Learn- 
ing Systems and the Work Keys system provide valuable information about individuals' basic 
skill levels. Together, they provide an "assessment continuum" that can be likened to an exten- 
sion ladder, with the Workforce Learning Systems assessments forming the lower ladder rungs, 
and the Work Keys system assessments extending beyond the Workforce Learning Systems' 
highest skill levels and forming the higher ladder rungs (see Figure 2. 7). 

There is a critical need for implementing a workforce focused assessment system that addresses 
this broad continuum of skill levels, from the "first rung of the ladder" (basic skills), up to the 
"top rungs of the ladder" (more advanced skills). The National Adult Literacy Survey, released 
by the U.S. Department of Education in 1993, categorized our nation's literacy skills into five 
levels.^ This study found that approximately half of the adult population is functioning at the two 
lowest levels (levels 1 and 2), nearly one-third at the middle level (level 3), and only about one- 
sixth at the highest two levels (levels 4 and 5). 

Some individuals functioning at the lowest NALS literacy level (level 1) are able to perform 
simple, routine tasks, like totaling an entry on a deposit slip, locating the time or place of a 
meeting on a form, and identifying a piece of specific information in a brief news article. Others 
at this level cannot even perform these tasks. Individuals functioning at the second lowest NALS 
literacy level (level 2) may be able to calculate the total cost of a purchase or determine the 
difference in price between two items, locate a particular intersection on a street map, and enter 
background information on a simple form. Those functioning at the middle NALS literacy level 
(level 3) are able to integrate information and determine appropriate arithmetic operations. 

Individuals functioning at the second highest NALS literacy levels (level 4) are able to synthe- 
size information from lengthy or complex passages, make inferences based on text and docu- 
ments, and perform sequential arithmetic operations using numbers found in different types of 
displays. Those functioning at the highest NALS level (level 5) are able to contrast complex 
information found in written materials, or make high level inferences or search for information 
in dense text; use specialized knowledge and search through complex displays for particular 
pieces of information; and determine the features of arithmetic problems either by examining 
text or by using background knowledge, and then perform the multiple arithmetic operations 
required. This broad range of skills, represented by the five NALS levels, must be matched by an 
equally broad system of skill assessments. 



^ See Kirsch, Jungeblut, Jenkins, and Kolstad, Adult Literacy in America: A First Look at the Results of the National 
Adult Literacy Survey (Washington, DC: National Center for Education Statistics, 1993) for more information on 
NALS and the NALS levels. 
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Figure 2.1 
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The Extending the Ladder project began with an assumption that CASAS' Workforce Learning 
Systems, which uses assessments and instructional materials tailored to the current skill levels of 
learners, will help individuals at the lower levels of the skills continuum begin to acquire the 
skills needed for a particular job or occupation. Individuals at the upper levels of the skills 
continuum need higher level assessments. The Extending the Ladder project began with the 
assumption that Work Keys system assessments would be better at measuring the more ad- 
vanced skills these individuals would have. 

The Extending the Ladder project assumed that individuals in the middle of the skills continuum 
might be well served by both assessment systems. Just as an extension ladder needs a solid, 
overlapping section to ensure its stability and structural integrity, the Workforce Learning Sys- 
tems and Work Keys system would benefit from an overlap in skills assessment, in the middle of 
the continuum, to provide a smooth transition from one assessment system to the next. A strong 
correlation over some middle skill range would provide a useful transition from lower level 
basic skills to higher level, more advanced skills. 

This study was designed to determine whether the Workforce Learning Systems and Work Keys 
system assessments can be linked to create an assessment scale extension ladder that can sup- 
port learners throughout the skills acquisition process. If the two systems can be linked in this 
way, they will help learners begin their studies at their current skill levels, and continue their 
studies until they have acquired the skills needed for the jobs they are seeking. An added benefit 
is the fact that workforce and worksite training programs will clearly see a solid link joining the 
Workforce Learning Systems and Work Keys system. 

Methodology 

The Extending the Ladder project team f/rsf conducted a content/cognitive review to verify the 
study's assumptions, and then conducted an empirical study to address the study's two research 
questions. 

Content/ Cognitive Review 



The project's content/cognitive review verified the project's five assumptions concerning: 

• The continuity of assessing basic and advanced workplace skills, 

• The Workforce Learning Systems' ability to assess basic skills, 

• The Work Keys system's ability to assess more advanced skills, 

• The two systems' content similarity and overlap in the range of skill levels covered, and 

• The existence of a complementary link between the two assessment systems. 
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For the content/cognitive review, CASAS and ACT staff exchanged assessments and related 
material for the four assessments under study: CASAS' Employability Competency System (ECS) 
reading and mathematics appraisals, and Work Keys' Reading for Information and Applied Math- 
ematics assessments. Staff from each organization then reviewed all test items, noted the con- 
tent and skills measured, identified the extent of the content/skills match between the CASAS 
and Work Keys instruments, shared their findings, and discussed the relationship between the 
two systems. 

In order for the Workforce Learning Systems and Work Keys system assessments to be linked, 
they need to measure similar content (reading and mathematics), and they need to contain 
some items measuring skills at comparable levels of difficulty (the area of overlap on the exten- 
sion ladder). The content/cognitive review found considerable similarity in the content mea- 
sured by the two systems, with both systems relating assessment items to "real-life" workforce 
tasks, as opposed to academic tasks. 

The ECS mathematics appraisal and Work Keys Applied Mathematics assessment measure very 
similar skill sets, while over 70 percent of the ECS reading appraisal measures skills similar to 
those found in the Work Keys Reading for Information assessment. The most significant content 
difference between the two assessment systems is that the ECS reading and mathematics ap- 
praisals test skills in interpreting graphs, while Work Keys tests these skills in a third assessment 
instrument, Locating Information, which was not part of this study. 

The content/cognitive review also discovered a number of structural differences between the 
two assessment systems. They contain a different number of items within each test (25 each for 
the ECS appraisals, 30 each for the Work Keys system assessments), they provide a different 
number of responses for each test item from which test takers can choose (four for ECS apprais- 
als, five for Work Keys system assessments), and they require a different length of time for 
completion (25 minutes for each ECS appraisal, 40 minutes for each Work Keys system assess- 
ment). The range of difficulty covered by the two systems' assessments also differ, but the review 
found enough of an overlap to meet the conditions for establishing an empirical link. 



Empirical Study 



Once the assumptions were verified, the project team designed and conducted an empirical 
study to address the project's two research questions on: 

• The degree of correlation in the range of skill levels covered by the two systems, and 

• The extent of the range of skill levels covered by the two systems in combination. 

For the empirical study, adult learners were asked to take both ECS and Work Keys reading 
assessments, and/or both ECS and Work Keys mathematics assessments. Comparisons were 
then made, for each individual, between scores on the paired tests in order to determine the 
linkage between the ECS and Work Keys system assessments. Results from those taking both 
pairs of tests were used to check the internal consistency of the ECS and Work Keys system 
instruments. 
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site Selection. Data were collected from 27 sites across eight states (See Appendix D for data 
collection information by state and site). States were selected for participation from across the 
country. Participating sites were selected from locations with workforce literacy programs, to 
ensure that the individuals participating in the assessments would represent the adult popula- 
tion in workforce literacy programs. 

Neither states nor sites were selected randomly. While many states volunteered to participate in 
the project, the funding and scope limitations of the study circumscribed the project bound- 
aries. Project sites included the variety of settings where adult workforce learning programs are 
provided, ranging from community colleges to correctional institutions, and from adult educa- 
tion to worksite training programs. 

Participant Sampling Procedures. Sites were asked to randomly select 20 to 30 learners to 
participate in the study. Efforts were made to ensure that the participants were representative of 
the total population (by sex, ethnicity, age, etc.), which required some sites to use a stratified 
random sampling procedure. To encourage participation, individuals were awarded dictionar- 
ies for taking the reading assessments and calculators for taking the mathematics assessments. 
The number of examinees with usable scores at each site ranged from two to 31 (see Appendix 
D). 

Instrumentation. In order to establish an empirical link between the two systems, the assess- 
ments had to be administered to the same examinees, and the examinees had to have skills at 
levels appropriate to the assessments. Individuals participating in the reading study took both 
the ECS form 130 reading appraisal and the Work Keys Reading for Information assessment. 
Those participating in the mathematics study took both the ECS form 130 mathematics ap- 
praisal and the Work Keys Applied Mathematics assessment. 

The two ECS appraisals took 25 minutes each, and the two Work Keys system assessments 
required 40 minutes each as indicated in Table 2.4. 



Table 2.4 

Time Required for Administering the Assessments 
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rk^essment In^tnimen^ ^ 


^ Minutes ^ : 


: Assessment Instrument , / " ' 


' Minutes ' 


CASAS Employability Competency System 




CASAS Employability Competency System 




Form 130 Reading Appraisal 


25 


Form 130 Mathematics Appraisal 


25 


ACT Work Keys Reading for Information 


40 


ACT Work Keys Applied Mathematics 


40 


TOTAL 


65 


TOTAL 


65 



C/IS/IS and ACT, 1997. 
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Instrument Administration. CASAS and ACT were responsible for mailing their assessment 
materials and instructions to state coordinators, who distributed the materials to the collection 
sites. A state coordinators' orientation conference call was held September 4, 1 996, to train the 
coordinators on test administration and sampling procedures. Once the assessments had been 
administered, data collection sites mailed both the ECS and Work Keys system assessments to 
the state coordinators, who forwarded the ECS assessments to CASAS and the Work Keys assess- 
ments to ACT. CASAS and ACT were responsible for scoring the individual assessments. 

Data Collection. A total of 551 people took one or both of the Workforce Learning Systems ECS 
appraisals, and 558 took one or both of the Work Keys system assessments. The test results of 
some of these individuals were removed from the study. This occurred for one of two reasons: 

• Examinees did not answer enough questions on the test to produce a reliable score, or 

• There was no corresponding test from the other testing organization, for the same 
individual, in the same subject to create a match. 

With these tests removed, the study had scores from 494 individuals (a reduction of 
approximately 1 0 percent from the number of initial test takers): 1 93 with only reading test 
scores, 1 63 with only mathematics test scores, and 1 38 with both reading and mathematics 
test scores as indicated in Table 2.5. 



Table 2.5 

Number of Examinees 
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Reading and ' 




Instrument 


Reading Only 


■ Mathematics Only 


Mathematics 


Totals, 


Workforce Learning Systems 


172 


168 


211 


551 


Work Keys 


211 


191 


156 


558 


Matched Workforce Learning Systems 








and Work Keys 


193* 


163 


138 


494 



*The number of matched Workforce Learning Systems and Work Keys reading scores exceeds the number of Workforce 
Learning Systems reading scores because some of the Work Keys reading scores were matched with Workforce Learning 
Systems reading scores from individuals who also took the Workforce Learning Systems mathematics assessment (those in 
column four). 

CASAS and ACT, 1997 

Delimitations. The sampling method used for this study imposed some limitations on the use of 
the study results. 

• The study results are useful in implementing a comprehensive assessment system including 
both the Workforce Learning Systems and Work Keys system. However, they are only relevant 
for the Workforce Learning System's ECS form 130 reading and mathematics appraisals and 
the Work Keys' Reading for Information and Applied Mathematics assessments, and not for 
any other Workforce Learning Systems or Work Keys assessments. 

• Because the participants in this study came from workplace and workforce development 
programs, recommendations from the study are most relevant for individuals in these types of 
programs. 
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• Study participants were only representative of a subset of the workplace and workforce 
development programs served by the Workforce Learning Systems and Work Keys system, 
and may not be representative of the universe of programs served by these two systems. 

• This study focused on reading and mathematics assessments. Both the Workforce Learning 
Systems and Work Keys system have additional system components to assist human resource 
development efforts, but linkages for these other system components were not included in 
this study. 

• The study participants were adults. Therefore, the results may not be readily applicable to 
programs serving younger individuals. 
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Chapter Three 



Research Results 



This chapter reports the results of the empirical study. It includes information about the examinees 
who participated and an analysis of their scores. The analysis was designed to address each of 
the research questions and, if the results indicated that linking could appropriately be 
accomplished, to actually link the CASAS Workforce Learning Systems and ACT Work Keys 
score scales. 

Collection of data for the empirical study went as planned. Individual adult students were asked 
to take the CASAS Workforce Learning Systems and the ACT Work Keys assessments for both 
reading and mathematics. Their answer documents were sent to the appropriate organization, 
CASAS or ACT, for scoring. Once the data records had been prepared for the individual examinees 
on the tests they took, these records were matched so that both the Workforce Learning Systems 
and the Work Keys scores appeared in each examinee's record. 

Since some individuals took only the reading assessments, some took only the mathematics 
assessments, and some took both the reading and the mathematics assessments, the total number 
of participants in the study is larger than the number of individuals who took any pair of 
assessments. There were some examinees who marked very few items and received scores at or 
below the chance level. In order to ensure that the results of the study were as accurate as 
possible, the records for these examinees were excluded from the analysis. Although this reduced 
the total number of examinees included in the data analyzed for each skill area, it was determined 
to be preferable to including data that might reduce the reliability or validity of the results. 

Characteristics of the Study Participants 

Sample sizes for the study depend on whether the original, unmatched numbers or the final 
matched sample are being considered. While a total of 558 individuals participated in the 
study, not all of them took all the tests. Based on the unmatchediWe, 383 individuals took the 
Workforce Learning Systems reading assessment and 367 individuals took the Work Keys reading 
assessment; 379 took the Workforce Learning Systems mathematics assessment and 347 took 
the Work Keys mathematics assessment. 

The number of cases was reduced by the matching process, since some records from Workforce 
Learning Systems and Work Keys did not match based on name, ID number, and other information. 
Exclusion of a few cases where individuals had marked very few items on a test reduced the 
sample a little further. Table 3.1 presents the sample sizes for the final matched file, which 
included a total of 494 examinees. All of the remaining parts of the analysis were performed on 
this group of 494 examinees. 
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Table 3.1 

Sample Sizes for the Workforce Learning Systems/ 
Work Keys Study Matched Cases 
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CASAS and ACT, 1997 



Note that the diagonal from top left to bottom right in Table 3.1 gives the remaining sample sizes 
for each of the individual tests while the other cells give sample sizes for pairs of tests. Some 
examinees took all four tests to provide data relating the two skill areas within a testing program. 
This allowed each program to compare that information with similar data from their typical 
examinees (see Appendix £). 

Table 3.2 provides demographic information about the study participants tested. Many study 
participants did not report all of their demographic data. About three-quarters of the study 
participants reported their gender; of these, about two-thirds were female and one-third male. 
Almost 80 percent reported information for computing age, and over 65 percent of these 
individuals were between the ages of 1 8 and 45. 

More than half of the study participants reporting an ethnic group were white. There were 
smaller percentages of Hispanic and black students and only a few students with Asian, Pacific 
Island, or Native American or Alaskan backgrounds. Most study participants chose to report 
their native language, and for about three-quarters of them it was English. About ten percent (51 
students) identified Spanish as their native language. Eleven additional languages were marked 
by 53 students, and an additional 19 students marked "other" as their native language. 
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Demographic 


Table 3.2 

Information for Study Participants 






Category -f 


Frequency ^ i 


Percent Valid Percent 








Gender 






Female 


236 


47.7 


62.4 




Male 


142 


28.7 


37.6 




Not Reported 


117 


23.6 










Age 






<18 


46 


9.3 


11.5 




18-25 


125 


25.3 


31.3 




26-35 


116 


23.4 


29.1 




36-45 


81 


16.4 


20.3 




46-55 


25 


5.1 


6.3 




>55 


6 


1.2 


1.5 




Not Reported 


96 


19.4 








Racial or Ethnic Group 






Asian or Pacific Islander 


22 


4.4 


6.3 




Black 


47 


9.5 


13.4 




Hispanic 


57 


11.5 


16.3 




Native American or Alaskan 


8 


1.6 


2.3 




White 


195 


39.4 


55.7 




Other 


21 


4.2 


6.0 




Not Reported 


145 


29.3 










Native Language 






Arabic 


5 


1.0 


1.0 




Cambodian 


2 


0.4 


0.4 




Chinese 


10 


2.0 


2.0 




English 


360 


72.7 


74.5 




Farsi 


3 


0.6 


0.6 




Japanese 


2 


0.4 


0.4 




Korean 


1 


0.2 


0.2 




Polish 


2 


0.4 


0.4 




Portuguese 


1 


0.2 


0.2 




Russian 


2 


0.4 


0.4 




Spanish 


51 


10.3 


10.6 




Tagalog 


2 


0.4 


0.4 




Vietnamese 


23 


4.6 


4.8 




Other 


19 


3.8 


3.9 




Not Reported 


12 


2.4 








Highest Diploma or Degree Earned 




None 


279 


56.4 


58.6 




GED 


25 


5.1 


5.3 




High School 


106 


21.4 


22.3 




Technical Degree 


18 


3.6 


3.8 




Associate Degree 


10 


2.0 


2.1 




Other 


38 


7.7 


8.0 




Not Reported 


19 


3.8 
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The majority of study participants had not attained any educational diploma or degree. Over 
one quarter had attained a GED or high school diploma, and slightly more than five percent had 
attained a technical or associate degree. 

Relationships Between Level Scores for the Two 
Assessments 

Research question one states: Are the Workforce Learning systems and Work Keys 
assessments in reading and mathematics sufficiently highly correlated in the range of skill 
levels covered to permit meaningful articulation of the two scales? 

This question focused on whether there was a sufficient empirical relationship between the 
different measures (Workforce Learning Systems and Work Keys) within a single skill area to 
support linking the scales. The absence of this relationship would mean that the measures are 
focused on different skills and that linking them would not be an appropriate approach to take. 

As described previously, the CASAS Workforce Learning Systems scale has five levels, with level 
A being the lowest and E the highest, and the ACT Work Keys tests have six levels with "below 
level 3" being the lowest, followed by levels 3 to 7, with level 7 being the highest. Most of the 
examinees who took each test scored toward the middle values of the score scale for that test. 
This is important because it means that the tests were at appropriate levels of difficulty for most 
examinees, which helped to ensure that the resulting scores would be valid. Had this not been 
the case, the data would not have provided the information necessary to support the linking. 

Tables 3.3 and 3.4 show the numbers of examinees receiving different combinations of CASAS 
Workforce Learning Systems and ACT Work Keys level scores for reading and mathematics skill 
areas, respectively. Correlation coefficients were computed between the two measures in each 
skill area. The resulting correlations were .71 for reading and .70 for mathematics. This means 
that the relationship between the tests accounted for about 49 percent of the variance in scores, 
a sufficiently strong relationship to support linking the scales. 





Table 3.3 




Number of Individuals at CASAS Workforce 




Learning Systems and ACT Work Keys Reading Levels 


•lAcXrn.:.:.: 


^ : CASAS Worl^oree Leafriihg3ystefm’t I ; 


~Wbrl< Keys 


» Level 1 Level B " 1 : Level C , Level D ^ Level E ^ i • I Jotal; : 



Below 3 


2 


23 


28 






53 


Level 3 




9 


36 


8 


2 


55 


Level 4 




2 


57 


60 


9 


128 


Level 5 




1 


7 


24 


26 


58 


Level 6 




1 


2 


2 


21 


26 


Level 7 








1 


10 


11 


Totals 


2 


36 


130 


95 


68 


331 



CASAS and ACT, 1 997 



Table 3.4 



Number of Individuals at CASAS Workforce Learning Systems 
and ACT Work Keys Mathematics Levels 



i-Acf ' ■■ 


.j' 


CASAS Workforce LWrhing System^ 




Work K^ys- 


* i'-LOVel-'A^';;-: 


;Levdl;B t ; 


:<Leyel C . 


i V j.Levef b 


)V Level E-:' 


:-I’'':-Totals ' . 


Below 3 


5 


39 


7 






51 


Level 3 




58 


45 


1 


1 


105 


Level 4 




24 


52 


15 




91 


Level 5 




3 


16 


15 


1 


35 


Level 6 






1 


7 


5 


13 


Level 7 








1 


5 


6 


Total 


5 


124 


121 


39 


12 


301 
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Although there is an empirical relationship that is strong enough to support the linking, Tables 
3.3 and 3.4 also show that there is not a one-to-one correspondence between the scores on 
Workforce Learning Systems and Work Keys assessments. This suggests that the two tests in 
each skill area are not measuring exactly the same thing with the same level of reliability. For 
this reason, scores on one assessment cannot be directly substituted for scores on the other. 
However, the relationship between the score scales is strong enough to permit scores on one to 
be estimated from scores on the other. 



Differentiation of the Two Scales 

Research question two states: Are the Workforce Learning Systems and Work Keys assessments 
in reading and mathematics different enough in the range of skill levels covered so that articulation 
of the two scales will extend beyond the range of skills covered in the individual assessments? 

When two scales are empirically related, it is possible to predict scores on one from scores on 
the other, and this is useful in some cases. To find the answer to research question 2, that is, to 
determine the amount that the ladder could be extended by linking the CASAS Workforce Learning 
Systems and ACT Work Keys score scales, it was necessary to begin by putting the Workforce 
Learning Systems and Work Keys reading assessments onto a common scale, and both 
mathematics assessments onto a common scale. 

Because esf/maf/on in both directions (from Workforce Learning Systems to Work Keys and from 
Work Keys to Workforce Learning Systems) was required, this process was completed twice, 
once by starting with the scale for the Workforce Learning Systems assessment and determining 
how the scale for the corresponding Work Keys assessment related to the CASAS Workforce 
Learning Systems scale; and once by starting with the ACT Work Keys scale and determining 
how the scale for the corresponding Workforce Learning Systems assessment related to it. 
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In order to achieve the greatest accuracy in linking, the metrics with the greatest definition 
(number of score points) were used. For Workforce Learning Systems these were scaled scores, 
and for Work Keys they were scaled item response theta scores. Because the relationships between 
each of these scores and the corresponding level scale for each assessment were known, it was 
then possible to see how the CASAS Workforce Learning Systems levels relate to the ACT Work 
Keys levels. 

The relationships, when starting with the CASAS Workforce Learning Systems scales and 
determining how the ACT Work Keys scales relate to them, are shown in figure 3.1 . The top bar 
in each section of the graphic shows the CASAS Workforce Learning Systems scale, and the 
bottom bar shows where the ACT Work Keys levels can, on average, be linked to that scale. 

To calculate this relationship, for each Workforce Learning Systems score point at which any 
examinee scored, the Work Keys theta score for every examinee with that Workforce Learning 
Systems score was found. The average of this subset of scaled scores was then calculated. For 
example, if there were 35 examinees with a CASAS Workforce Learning Systems scaled score of 
226, the ACT Work Keys theta score for each of these examinees was identified, and the average 
of these 35 scores was calculated. 

This averaging process was repeated for every CASAS Workforce Learning Systems scaled score 
achieved by any examinee who took both tests in the study. This created a correspondence 
between each Workforce Learning Systems score on the CASAS scale and the average Work 
Keys theta score for every Workforce Learning Systems score attained by at least one study 
participant. 

Mapping the Work Keys level scale onto the CASAS scale was then achieved by matching the 
Work Keys theta scores at the end points of each Work Keys level with their corresponding 
Workforce Learning Systems scores on the CASAS scale. For example, the Work Keys theta 
score at the low end of Work Keys level 3 corresponded with a score of 229 on the CASAS 
reading scale. Thus, the low end of Work Keys level 3 was aligned with 229 on the CASAS 
reading scale in figure 3.1 . 

Because this relationship is based on an average score for examinees scoring at 229 on the 
CASAS reading scale, an examinee who scored a 229 on the CASAS reading scale would be 
expected to have about a 50/50 chance of scoring at Work Keys level 3 in reading. That is, this 
relationship is accurate for group averages, but does not provide unambiguous information 
about individual examinees' scores. 

If showing the relationships between the scales has the desired effect of extending the scale, it 
should be possible to see a contrast between CASAS Workforce Learning Systems and ACT 
Work Keys in the differentiation of levels at one end of the scale. That is, one would expect that 
at the lower end of the skill scales, the CASAS Workforce Learning Systems scale would 
differentiate several levels where the ACT Work Keys scale does not. 

This is in fact the case. For reading, all of CASAS Workforce Learning Systems level A and most 
of level B are below ACT Work Keys level 3. For mathematics, all of CASAS Workforce Learning 
Systems level A and about half of the lower portion of level B (Beginning Basic Skills) are below 
ACT Work Keys level 3. This shows that Workforce Learning Systems is providing the desired 
extension below Work Keys level 3 in both skill areas. 




Figure 3.1 
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The relationships, when starting with the ACT Work Keys scales and determining how the CASAS 
Workforce Learning Systems scales relate to them, are shown in figure 3.2. The top bar in each 
section of the graphic shows the ACT Work Keys scale, and the bottom bar shows where the 
CASAS Workforce Learning Systems levels can, on average, be linked to that scale. In this case, 
the average CASAS Workforce Learning Systems scaled scores of all of the examinees at each 
ACT Work Keys theta score were calculated. This created a correspondence between each 
Work Keys theta score and the average Workforce Learning Systems scores on the CASAS scale 
for every Work Keys theta score attained by at least one study participant. 

Mapping the CASAS scale onto the Work Keys level scale was then achieved by matching the 
Workforce Learning Systems scaled scores at the end points of each CASAS level with their 
corresponding Work Keys theta scores on the Work Keys scale. For example, the high end of 
CASAS level B and low end of CASAS level C is between 220 and 221 on the CASAS scale. In 
figure 3.2, this point on the CASAS scale was mapped onto its corresponding Work Keys theta 
score, which, for mathematics, is about two-thirds of the way into Work Keys level 3. 
Corresponding theta scores were found for the end points of each CASAS level, in order to show 
how the CASAS Workforce Learning Systems levels would relate, on average, to the ACT Work 
Keys levels. 

Again, if identifying the relationships between the scales for each skill area has the desired effect 
of extending the scales, it should be possible to see a contrast between Work Keys and Workforce 
Learning Systems in the differentiation of levels at the top end of the scales. That is, one would 
expect that at the upper end of the score scales, the ACT Work Keys scales for reading and 
mathematics would differentiate several levels where the CASAS Workforce Learning Systems 
scales do not. 

This also appears to be the case. For reading, where ACT Work Keys distinguishes between level 
5, level 6, and level 7, CASAS Workforce Learning Systems level E corresponds to both levels 6 
and 7, and to part of level 5 as well. In mathematics, CASAS Workforce Learning Systems level 
D corresponds to a small portion of the top of ACT Work Keys level 5 and about the first third of 
level 6, and CASAS Workforce Learning Systems level E corresponds to the remainder of ACT 
Work Keys level 6 and a significant part of level 7. The Work Keys scale, then, does provide 
additional differentiation at the top end of both skill scales. 
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Figure 3.2 
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Caveats 



Appropriate use of the Workforce Learning SystemsAVork Keys score articulation depends on 
the circumstances of the examinee and on the intended use of the score. For many reasons, it is 
not appropriate to simply substitute either score for the other. Neither test is perfectly reliable, 
and the correlation between them is also less than perfect. For any given examinee, the relationship 
between the two test scores may not, as a result, be exactly the same as the average relationship 
between the scores. 

The examinees were not fully representative of either Workforce Learning Systems or Work Keys 
examinee populations, and some of them may not have been highly motivated to achieve on 
one or both tests. This could lead to anomalies in the data, especially at the ends of the scale 
where relatively few examinees scored, as a single unusual response there could have a greater 
impact on the average relationship between the scales. Therefore, in high-stakes decisions and 
other situations demanding the most accurate information, each individual should use the 
assessment that is most appropriate to his or her particular situation. 

Applying the Results: How to Interpret the Information 
from this Study 

The Workforce Learning Systems and Work Keys programs can be articulated to more fully meet 
the needs of adult learners. Just how their articulation can be used depends on the level at 
which an individual enters the program and the intended use of the score. By articulating these 
two systems, it is possible to allow individuals to be assessed upon entering an instructional 
program at whatever level they are currently functioning, and to provide assessment to support 
their eventual transitions into the workplace as they reach the skill levels required by employers. 

The next four tables (Tables 3.5 to 3.8) may be used to esf/mafe what level a person with Workforce 
Learning Systems scores might achieve on the Work Keys assessments, and what level a person 
with Work Keys scores might achieve on the Workforce Learning Systems assessments. They 
have been designed to help practitioners use the study results to provide learners, employees, 
and others who have scores on one assessment with information about how they might score on 
the other. 

There are two tables for the reading assessments (Tables 3.5 and 3.7): one based on the CASAS 
Workforce Learning Systems scale (Table 3.5) and the other based on the ACT Work Keys scale 
(Table 3.7). There are also two similar tables for the mathematics assessments (Tables 3.6 and 
3.8). All of the tables are to be read in the same way. Start by looking in the leftmost column for 
the score level that matches the score the person already has. Then, read across that row to find 
the chances in 1 00 (the probability) that the individual would have each of the different level 
scores on the other test for the same skill area. 

Table 3.5, for example, shows that if an individual has an ACT Work Keys reading score level 4, 
he or she has 2 chances in 1 00 of having a CASAS Workforce Learning Systems reading level of 
B, 44 chances in 100 of having a CASAS Workforce Learning Systems reading level of C, 47 
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chances in 100 of having a CASAS Workforce Learning Systems reading level of D, and 7 
chances in 1 00 of having a CASAS Workforce Learning Systems level of E. That is, if this person 
were to take the Workforce Learning Systems reading test, he or she would probably score at the 
C or D level. 

These tables are more complete than the graphic scales shown in Figures 3.1 and 3.2. They 
make it clear that a level on either test is related to more than one level on the other. The scales 
shown in Figures 3.1 and 3.2 do not show this information because they are based on average 
scores. These tables make it clear that a level on either test is related to more than one level on 
the other. The information presented in these four tables, along with other information about 
examinees, can help educators, training specialists, and counselors make better instructional 
decisions. 

These tables also provide one additional piece of useful information: the number of examinees 
in this study at each level. For example. Table 3.5 shows that for Work Keys reading level 4 there 
were 128 examinees in the study. One person represents 1/128 or about .7 percent of the 
sample. The percentages based on 1 28 examinees should be reasonably stable because it would 
take quite a few examinees scoring in a particular way to change the percentage at a given level. 

However, observe the Work Keys reading score at level 7. With only J1 examinees in the study 
on which to base percentages, the score of a single examinee can make quite a difference. One 
examinee represents 1/11 of the sample or about 9 percent. Therefore, we can know that one of 
the examinees who scored at level 7 in reading on Work Keys scored at level D on Workforce 
Learning Systems, and the other ten scored at level E. With so few examinees at a level, the 
percentages at these levels must be interpreted with greater caution. 



Table 3.5 

Ejipected CASAS Workforce Learnisig Systems Reading Levels 
as Indicated by ACT Work Keys Scores 



ACT Work Keys 
Reading for 
Information Score 



Below 3 (N=53) 
Level 3 (N=55) 
Level 4 (N=1 28) 
Level 5 (N=58) 
Level 6 (N=26) 
Level 7 (N=11) 



Chances in 100 of each CASAS Workforce Learning 
Systems Reading Level 

|~ Level A 



Level B 


Level C 


Level D 


Level 


43 


53 






16 


66 


14 


4 


2 


44 


47 


7 


2 


12 


41 


45 


4 


8 


8 


81 






9 


91 



Note: The rows may not sum to 1 00 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1 997 
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Table 3.6 

Expected CASAS Workforce Learning Systems Mathematics Levels 
as Indicated by ACT Work Keys Scores 



ACT Work Keys 
Applied 


Chances in 100 of each CASAS Workforce Learning 
Systems Mathematics Level 




Mathematics Score 


Level A 


Level B 


Level C 


Level D 


Level E 


Below 3 (N=51) 


10 


76 


14 






Level 3 (N=105) 




55 


43 


1 


1 


Level 4(N=91) 




26 


57 


16 




Level 5 (N=35) 




9 


46 


43 


3 


Level 6 (N=1 3) 






8 


54 


38 


Level 7 (N=6) 








17 


83 



Note: The rows may not sum to 100 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 



Table 3.7 

Expected ACT Work Keys Reading Levels as Indicated by 
CASAS Workforce Learning Systems Scores 



CASAS Workforce 
Learning Systems 
Reading Score 


Chances in 100 of each ACT Work Keys 
Reading for Information Level 






Below 3 


Level 3 


Level 4 


Level 5 


Level 6 


Level 7 


Level A (N=2) 


100 












Level B (N=36) 


64 


25 


6 


3 


3 




Level C (N=121) 


22 


28 


44 


5 


2 




Level D (N=39) 




8 


63 


25 


2 


1 


Level E (N=1 2) 




3 


13 


38 


31 


15 



Note: The rows may not sum to 1 00 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 



Table 3.8 

Expected ACT Work Keys Mathematics Levels as Indicated by 
CASAS Workforce Learning Systems Scores 



CASAS Workforce 
Learning Systems 
Mathematics Score 



Chances in 100 of each ACT Work Keys 
Applied Mathematics Level 



Below 3 Level 3 



Level 4 Level 5 



Level 6 Level 7 



Level A (N=5) 
Level B (N=1 24) 
Level C(N=1 21) 
Level D (N=39) 
Level E (N=^ 2) 



100 

32 

6 



47 

37 

3 

8 



19 

43 

38 



3 

13 

38 

8 



1 

18 

42 



3 

42 



Note: The rows may not sum to 1 00 due to rounding. Column totals are not meaningful. 
CASAS and ACT, 1997 
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Summary of Major Findings 

The relationships between the CASAS Workforce Learning Systems and ACT Work Keys scales 
do offer the anticipated extension ladder effect, with CASAS Workforce Learning Systems 
extending below and differentiating more levels at ACT Work Keys level 3 in both skill areas, 
and ACT Work Keys differentiating more levels at the upper end and extending above CASAS 
Workforce Learning Systems. Therefore, the CASAS Workforce Learning Systems and ACT Work 
Keys scales can be articulated, with the effectiveness of their use depending on the level at 
which a student enters the program and on the intended use of the score. 

Although the relationship between the score scales for each skill area is strong enough to permit 
scores on one test to be estimated from scores on the other, scores on one test may not be 
directly substituted for scores on the other for the same skill area. 

Tables 3.5 to 3.8 may be used to esf/mafe what level a student with Workforce Learning Systems 
scores might achieve on the Work Keys assessments, and what level a student with Work Keys 
scores might achieve on the Workforce Learning Systems assessments. 
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Chapter Four 



Conclusions, 

Recommendations, 
and Potential Uses 



This chapter reports the conclusions, recommendations and potential uses of the study results. 
The conclusions were based on the results presented in Chapter Three. The recommendations 
were based on the conclusions. They are designed to be of assistance to educators, trainers, and 
human resource managers. The "Potential Uses" section presents some practical scenarios for 
utilizing the results of this study. 



Conclusions 

• Both CASAS' Workforce Learning Systems and ACT's Work Keys system measure basic and 
more advanced skills needed for effective participation in the workforce and workplace. 

• There is enough commonality \r\ both the content and range of skills covered by the Workforce 
Learning Systems and Work Keys system to determine a meaningful connection between the 
two. 

• There is enough difference in the range of skills covered by the Workforce Learning Systems 
and Work Keys system to warrant joining the two together in a basic skills continuum. This 
continuum provides more useful information over a greater range of skill levels than is avail- 
able from either system alone. 

• The Workforce Learning Systems covers skills that are more basic, and provides more infor- 
mation on the skills of individuals functioning at the lower end of the skills continuum. 

• The Work Keys system covers skills that are more advanced, and provides more information 
on the skills of individuals functioning at the higher end of the skills continuum. 

• The Workforce Learning Systems and Work Keys system complement each other, and can be 
used together to provide a smooth, progressive, and complete skills continuum. 

• A statistical relationship exists between the CASAS Workforce Learning Systems and the ACT 
Work Keys system scales that can be used to estimate scores on one assessment from scores 
on the other. 

• It is not appropriate to substitute scores on one assessment with those from the other. 
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Recommendations 



• Educators, trainers, and human resource managers can use the Workforce Learning Systems 
and the Work Keys system in tandem to help individuals progress along a broad skills con- 
tinuum, from their current skill levels to the skill levels required for successful participation in 
the workforce. 

• Educators, trainers, and human resource managers can use the Workforce Learning Systems 
assessments with individuals whose skills are on the lower end of the skills continuum. 

• Educators, trainers, and human resource managers can use the Work Keys system assess- 
ments with individuals whose skills are on the higher end of the skills continuum. 

• Educators, trainers, and human resource managers can use the assessments of either the 
Workforce Learning Systems, the Work Keys system, or the two systems together with indi- 
viduals whose skills are in the mid range of the skills continuum. 



Potential Uses 

For Those Using the Workforce Learning Systems 



• Individuals who have already been tested with a Workforce Learning Systems assessment can 
use the tables in this publication to estimate how they would perform on a Work Keys assess- 
ment, and determine whether it would be worthwhile for them to take that assessment now. 
This would be valuable in cases where the individuals assessed with Workforce Learning 
Systems needed to meet a Work Keys standard, perhaps for a new job or a promotion. Indi- 
viduals with a low probability of being able to score at the desired level should pursue further 
training before taking a Work Keys assessment. Individuals with a high probability of scoring 
at the desired level would not need to wait to take a Work Keys assessment. 

For example. Table 3.7 shows that an individual assessed at level B on the CASAS reading 
scale would have a 12 percent chance of scoring at or above level 4 on the Work Keys 
Reading for Information assessment (6% chance of scoring at level 4 + 3% chance of scoring 
at level 5 + 3% chance of scoring at level 6). Individuals needing to score at level 4 in order 
to be eligible for a specific job might be well advised to engage in more study before taking 
a Work Keys assessment. 

• Individuals in CASAS-based education and training programs can use Work Keys Occupa- 
tional Profiles, along with the tables in this publication, to estimate the CASAS scale scores 
they would need in order to meet the reading and mathematics requirements for specific 
occupations. Work Keys has developed profiles on a number of occupations by combining 
the results of occupational job analyses conducted in specific companies. 
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For example, a Work Keys Occupational Profile for licensed practical nurses shows that 
individuals interested in pursuing that occupation would need a score of level 6 on the Work 
Keys Reading for Information assessment. Table 3.7 shows that individuals at CASAS level A 
have no chance of meeting this standard, while those at CASAS levels B, C, and D have only 
a very small chance of meeting this standard (3% for level B, 2% for level C, and 3% (2% plus 
1 %) for level D).^ Because those scoring at CASAS level E have a 46 percent chance (31 % + 
15%) of meeting this standard, individuals interested in pursuing this job should probably 
continue their studies until they are able to achieve a score at the upper end of CASAS level E. 

• Individuals can also use the results from this study, along with their Workforce Learning 
Systems scores, to gauge their progress toward meeting a Work Keys level. Table 4.1 shows 
the CASAS scale score at which an examinee has a 50-50 chance of scoring at a specified 
Work Keys level. 



Table 4.1 

CASAS Workforce Learning Systems Scale Score at which an Examinee 
Has a 50-50 Chance of Scoring at a Specified ACT Work Keys Level 


Workforce 




Workforce 


Learning Systems 
Reading Score 


Work Keys Level 


Learning Systems 
Mathematics Score 


229 


3 


221 


238 


4 


230 


250 


5 


237 


260 


6 


244 


N/A 


7 


258 



N/A - Data not sufficient to estimate this number. 
CASAS and ACT, 1997 



Table 4.1 shows, for example, that individuals with Workforce Learning Systems mathemat- 
ics scores of 221 have a 50-50 chance of testing at Work Keys level 3 on the Applied Math- 
ematics assessment. Those with Workforce Learning Systems mathematics scores of 230, 
have a 50-50 chance of testing at Work Keys level 4. Individuals who raise their Workforce 
Learning Systems mathematics scores from 221 to 230 have, therefore, moved from a prob- 
able Work Keys level 3 to a probable Work Keys level 4. 



^As mentioned earlier, the small number of participants scoring at the extremes of the CASAS Workforce Learning 
Systems and ACT Work Keys system scales makes the study results less reliable in these areas. This is why table 3.7 
suggests that those at CASAS level C are slightly less likely to score at Work Keys level 6 than are those at CASAS 
level B. 
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• Individuals scoring in CASAS level E can gain further information on their skills and abilities 
by taking the Work Keys assessments. Moving from the Workforce Learning Systems to the 
Work Keys system enables these individuals to take advantage of the upward extension of- 
fered by the Work Keys system. 



For Those Using Work Keys 

• Individuals who have already been tested with a Work Keys assessment can use the tables in 
this publication to estimate how they would perform on a Workforce Learning Systems as- 
sessment, and determine whether it would be worthwhile for them to take that assessment 
now. This would be valuable in cases where the individuals assessed with Work Keys needed 
to meet a Workforce Learning Systems standard. Individuals with a low probability of being 
able to score at the desired level should pursue further training before taking the Workforce 
Learning Systems test. Individuals with a high probability of scoring at the desired level would 
not need to wait to take the Workforce Learning Systems test. 

For example, Table 3.6 shows that an individual with a Work Keys Applied Mathematics 
score of level 3 would have a 45 percent chance of scoring at or above CASAS level C on the 
Workforce Learning Systems reading assessment (43% chance of scoring at level C + 1% 
chance of scoring at level D + 1 % chance of scoring at level E). Individuals needing to score 
at CASAS level C might choose to study a little longer, or take the Workforce Learning Sys- 
tems assessment right away. 

• Individuals in Work Keys-based education and training programs can use the CASAS Materi- 
als Guide, along with the tables in this publication, to select appropriate curriculum and 
instruction materials. The CASAS guide correlates over 1,000 commercially available in- 
structional textbooks, videos, and software programs to more than 300 competency state- 
ments (or learning objectives). 

For example. Table 3.5 shows that individuals with an ACT Work Keys reading score of level 
3 have a 66 percent chance of scoring at CASAS Workforce Learning Systems level C. These 
individuals should select instructional materials in the CASAS guide geared toward CASAS 
level C. Table 3.6 shows that individuals with an ACT Work Keys mathematics score of level 
6 have a 54 percent chance of scoring at CASAS Workforce Learning Systems level D. These 
individuals should select mathematics instructional materials geared toward a CASAS level 
D. 

• Individuals in Work Keys-based education and training programs can use CASAS research to 
estimate how long it would take them to move from one Work Keys level to the next. Re- 
search on individuals within the CASAS system has shown an average increase of 5 points 
along the CASAS scale for every 1 00 hours of instruction. 

Table 4.2 combines the results from this study showing the relationship between the Workforce 
Learning Systems and Work Keys scores, and CASAS research indicating the average amount 
of time needed to advance along the CASAS scale. This provides estimates of the amount of 
time individuals would need to advance from one Work Keys level to the next. 
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Table 4.2 

Estimated Hours of instructiosu Needed 
to Move from One Work Keys Level to the Next 




Work Keys 


HOURS OF INSTRUCTION 




Assessment 


From Level 


From Level 


From Level 


From Level 


Instrument 


3 to 4 


4 to 5 


5 to 6 


6 to 7 


Reading for Information 


120 


180 


240 


N/A 


Applied Mathematics 


100 


180 


140 


140 



Based on CASAS research on Individuals using CASAS assessment Instruments. Hours are estimated based on moving from the 
beginning oi one level to the beginning oi the next level. 

N/A - Data not sufficient to estimate this number. 

CASAS and ACZ 1997 



The estimates in Table 4.2 will be most accurate for individuals typical of those served by 
CASAS (e.g., individuals in adult basic education), and in classes typical of those where 
CASAS assessments are used (e.g., classes of standard size with heterogeneous students). 
Initial information from industry-based training in which individuals scoring at the same Work 
Keys level were provided targeted instruction in the Work Keys skills suggests that Work Keys 
level advances can be made with as little as 25 hours of instruction. These results require 
highly motivated students with the prerequisite skills to benefit from the training; well de- 
signed, contextualized materials; and skilled instructors. 

• Individuals scoring below Work Keys level 3 can gain further information on their skills and 
abilities by taking the Workforce Learning Systems assessments. Moving from the Work Keys 
system to the Workforce Learning Systems enables these individuals to take advantage of the 
downward extension offered by the Workforce Learning Systems. 

For Those Using neither the Workforce Learning Systems nor Work Keys 

Selecting the most appropriate assessment to use with an individual should depend first on the 
purpose of the assessment. If, for example, the purpose is to compare the individual's score with 
a standard based on a Work Keys job profile, a Work Keys assessment is required. If, on the 
other hand, the purpose of the assessment is to provide the individual with appropriate instruc- 
tion based on the CASAS Instructional Materials Guide, a Workforce Learning Systems assess- 
ment is required. 

If the purpose of the assessment does not suggest one assessment over the other, the results of 
this research can provide guidance on which assessment system would be most appropriate for 
a particular individual. 

• Individuals likely to score below level 3 on the Work Keys assessments would be better 
served by the Workforce Learning Systems. Individuals likely to fall into this category in- 
clude: 
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- Those who have not completed formal education beyond the eighth grade, and are be- 
lieved to be functioning at or below this level, and 

- Those with a limited command of English who are believed to be functioning at or below 
the eighth grade level. 

• Individuals likely to score at CASAS level E on the Workforce Learning Systems assessments 
would be better served by the Work Keys system. Individuals likely to fall into this category 
include; 

- Those who have a high school diploma or GED, have completed at least one year of 
postsecondary education, and are believed to be functioning at or above a high school 
graduate level, and 

- Those who have worked successfully in a job requiring a Work Keys skill level of 6. 

For Those Seeking a Comprehensive Assessment System 

The results of this study can benefit the assessment efforts of education and training programs at 
state and local levels. Together, the Workforce Learning Systems and Work Keys system assess 
the broad range of skills programs are likely to encounter. By using these two systems in tan- 
dem, states and localities can operate a comprehensive, articulated assessment system, rather 
than a potpourri of unrelated assessment tests. 

Together, the Workforce Learning Systems and Work Keys system form a solid core for a program's 
assessment efforts. By using the Workforce Learning SystemsAVork Keys system assessments as 
a base, education and training programs will be able to ensure that any additional assessment 
instruments they use, such as interest inventories, are appropriate for the skill levels of the 
participants involved. 

• The results of this study are particularly useful for states and localities establishing one-stop 
systems. One-stop facilities are designed to serve any individuals needing employment and 
training services, from welfare recipients to dislocated workers. With such a broad mandate, 
one-stop centers need a comprehensive assessment system appropriate to all comers. 

The Workforce Learning Systems and Work Keys system meet this need. Together, the systems 
can assess individuals along the entire range of the skills continuum. The Workforce Learning 
Systems can be used with individuals with limited English proficiency, and even learning 
disabilities, as well as those with low to mid-level basic skills. The Work Keys system, on the 
other hand, can assess individuals with more advanced skills, including those with technical 
and managerial experience. 
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• The results of this study will also help states and localities meet their responsibilities under 
the new Welfare Reform mandate. Many welfare recipients will begin their education and 
training programs at skill levels most appropriately measured by the Workforce Learning 
systems. Many welfare recipients will then need to continue their education and training 
programs (before or after employment) until they reach skill levels best measured by the 
Work Keys system. 



Epilogue 

The overarching reason for conducting this study was to provide a unique and innovative view 
of assessment in the workplace. This study links CASAS' Workforce Learning Systems and ACT's 
Work Keys system to provide education and training programs with a more extensive and com- 
prehensive assessment capability. The effective use of these two systems, functioning in tan- 
dem, can identify training needs at the skill level of the individual and support continued train- 
ing to the requisite levels mandated by business and industry for occupations requiring higher 
skill levels. 

In the accelerated labor market of the twenty-first century, business and industry will constantly 
be looking for a highly trained workforce. The effective use of relevant assessment tools will 
increase companies' probability of gaining the greatest return on investment for the training 
dollars spent on the workforce. The effective use of the two assessment systems linked in this 
study will help secure a high return on investment for continued training and upgrading of the 
American workforce. 
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Appendix A 



List of Participating States and 
Data Collection Sites 



PARTICIPATING STATE 


CONTACT PERSON 


DATA COLLECTION SITE 


California 


Jane Eqiiez 


Ann Marie Damrau 




CASAS 


San Diego Community College 




8910 Clairmont Mesa Blvd. 


Center City - VESL Coordinator 




San Diego, CA 92123-1104 


1400 Park Blvd. 

San Diego, CA 92101 



Marjorie Knowles 
Mission College - 
3000 Mission Coll. Blvd. MS #1 
Santa Clara, CA 95054-1 897 

Sylvia Ramirez 
Mira Costa College 
320 N. Home St. 

Oceanside, CA 92054 



Connecticut Jim Harrison Judith Baldwin 

Connecticut Dept, of Education Area Cooperative Educational Services 

25 Industrial Park Road 5 Science Park 

Middletown, CT 06457 New Haven, CT 06511 

jay Cretella 

Wallingford Adult Education 
37 Hall Avenue 
Wallingford, CT 06492 

Nick Lavorato 

Applied Engineering Products 
1 04 John W. Murphy Drive 
P.O. Box 510 
New Haven, CT 065 1 3 

Reina Marasco 

Valley Regional Adult Education 
415 Howe Avenue 
Shelton, CT 06484 

David Talbot 
EASTCONN 

Northeast Learning Center 
111 Connecticut Mill Ave. 

Danielson, CT 06239 
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' PARTICIPATING STATE ! CONTACT PERSON ; DATA COLLECTION SITE T . : 

^ « ;o- <. ^ .... IS. , .. ,, .,j ,, .„. , : ,, :: ;r;; i : : : : " : ^ ^ -:' ^ =- •■ •^- - - •. 4- 

t - 5? 4 -if I ,,„,v_^: V .,, , : ,;„ ;,j.,ll .: r.4 ! !..;' : .": .! ,!. ! :f..'^. !■ ^ t s #4-5j- 



Iowa 



John Hartwig 

Iowa Department of Education 
Grimes State Office Bldg. 

Des Moines, lA 50319-0146 



Cindy Burnside 

Indian Hills Community College 
525 Grandview 
Ottumwa, lA 50801 

Mary Entz 

Des Moines Area Comm. College 
2006 S. Ankeny Blvd., Bldg. 18 
Ankeny, lA 50021 

Peg Garrison 
Jim Schneider 

Eastern Iowa Comm. Coll. Dist. 
627 W. 2nd 
Davenport, lA 52801 

Marty Lundberg 

North Iowa Area Comm. College 
500 College Drive 
Mason City, lA 50401 

Kay Nebergall 

Kirkwood Community College 
Lincoln Learning Center 
P.O. Box 2068 
Cedar Rapids, lA 52406 



Kansas 



Dianne S. Glass 
Kansas Dept, of Education 
120 East 10th Street 
Topeka, KS 66612 



Charlotte Hearn 

Fort Scott Community College 

2108 South Horton 

Fort Scott, KS 66701 

Janice Kelly 

Wichita Area Technical College 
301 S. Grove 
Wichita, KS 67211 

Phil Wegman 

Johnson County Comm. College 
1 2345 College at Quivers Road 
Overland Park, KS 66210 




54 



83 



PARTICIPATING 


state contact person i 


: : DATA COLLECTION SITE 


Maryland 


Beth Anagnostopoulos 
Program - USWA - Bethleham 
Steel Career Develop. Prog. 
Sparrows Point Division 
Sparrows Point, MD 21 21 9 


Program - USWA - Bethlehem 
Steel Center Develop. Program 
Sparrows Point Division 
Sparrows Point, MD 21 21 9 


Minnesota 


Brian Kanes 
Adult Basic Education 
Minnesota Dept, of Education 
995 Capitol Square Bldg. 

St. Paul, MN 55101 


Faith McCaghy 
Community Education Bldg. 
8755 Upper 208th St. West 
Lakeville, MN 55044 

Dawn Neton 

Alternative Learning Center 
223 East 7th Street 
Crookston, MN 56716 






Faye Petersen 
Carver- Scott Cooperative 
401 East 4th Street 
Chaska, MN 55318 






Lynn Swanson 
Metro East Adult Basic Ed. 
Ronald H. Hubbs Center 
for Lifelong Learning 
1 030 University Ave. West 
St. Paul, MN 55104-4706 



Nebraska 



Connie Eichhorn 
Omaha Public Schools 
321 5 Cuming St. 

Omaha, NE 68131-2024 



Omaha Public Schools 
3215 Cuming St. 

Omaha, NE 68131-2024 
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PARTiaPATING STATE 



ACT PERSON 



- ' ; ' 




itiON SITE ;■ ! 

i c:'" 



Oregon 



Sharlene Walker 
Oregon Dept, of Education 
255 Capitol St.; NE 
Salem; OR 97310-0203 



Pat Amsberg 

Blue Mountain Comm. College 
P.O. Box 100 
Pendleton; OR 97801 

Dawn DeWolf 

Oregon Coast Comm. College 
332 S.W. Coast Highway 
Newport; OR 97365 

Mary Louise Doran 
Rogue Community College 
3345 Redwood Highway 
Grants PasS; OR 97527 

Linnell Rantapaa 
Oregon State Corrections 
Education Division 
2575 Center St. NE 
Salem; OR 97310 

Leslie Rasor 

Lane Community College 
4000 East 30th 
Eugene; OR 97405 
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Proposal Letter to Dr. Andrew Hartman 



May 1, 1996 



Dr. Andrew Hartman 
Executive Director 
National Institute for Literacy 
800 Connecticut Avenue, NW 
Suite 200 

Washington, D.C. 20202-7560 
Dear Andy: 

Pursuant to our recent conversation at the State Directors' meeting, I am sending a follow-up 
letter regarding a proposed research study. The attached draft proposal, which is still 
evolving, was developed by Dr. Joel D. West, executive director, ACT Center for Education 
and Work and Ms. Patricia L. Rickard, executive director of CASAS. The overall purpose of 
this study is to examine the relationship between two national assessment systems, (CASAS 
and Work Keys) and determine where. the linkages between the two systems may be 
strengthened. 

I have discussed this research proposal with key state directors who perceive this type of 
study is needed with the advent of block grant legislation. I believe the proposal outlines the 
overall strategies, intent and purposes of the study. 

The following is a listing of states who have committed to participate in the study including 
the proposed number of data collection sites within each state. 

Participating Number 

State of Sites 



Iowa 


5 


Kansas 


3 


Minnesota 


3 


Oregon 


2-4 


California 


3 


Connecticut 


3-5 


Nebraska 


1 


TOTAL 


20-23 
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Dr. Andrew Hartman 
May 1 , 1 996 
Page 2 



Each data collection site would assess 1 5 to 20 examinees, bringing the total number of 
examinees within a range of 400 to 450. Pat, Joel and I perceive this would be an adequate 
number of examinees to constitute a national sample. 

The project is seeking funding from the National Institute for Literacy in the amount of 
$1 2,000 to $1 5,000 to support the research. Specifically, this funding will cover the following 
expenses: (1 ) qualified proctors or teachers to administer the assessment instruments, 

(2) provide each examinee with a pocket calculator for taking the mathematics assessment or 
a pocket dictionary for taking the reading assessment. This strategy will provide a motivation 
for examinees to perform at optimum levels. As the proposal indicates, the data collection 
phase of the project is scheduled for mid-September through late October 1 996. Therefore, 
there is time for discussion or to answer further questions. 

On a personal note, I perceive that participating states know this study is essential as the 
results will assist practitioners to accurately assess target populations to develop work force 
capabilities. The timing for this study is excellent due to the demand for accountability and 
performance standards. The results will assist states to develop effective assessment strategies. 

If you need further information please let me know. I will be looking forward to hearing from 
you. 

Sincerely yours. 



John Hartwig, Ph.D. 

CASASAA/ork Keys Project Director 

JH/bse 

cc: Patricia L. Rickard, Linda Taylor, Jane Eqiiez: CASAS - CA 
Dr. Joel D. West: ACT - lA 
Jim Harrison - CT 
Dianne S. Glass - KS 
Brian Kanes - MN 
Connie Eichhorn - NB 
Dr. Sharlene Walker - OR 

Marty Lundberg, Peg Garrison, Jim Schneider, Kay Nebergall, 
Mary Entz, Cindy Burnside, Christine Case, Mary Strom - lA 
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Appendix C 



Sample Assessment Items Referencing Reading and 
Mathematics for CASAS Levels A and E and Work Keys 
Levels 3 and 7 
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Sample Reading Item at CASAS Level A, <200 



Competency 4.1 .3: Identify and use sources of information about job opportunities such as job 
descriptions, job ads, and announcements, and about the workforce and job market. 



CASHIER 
MUST BE 21 
APPLY IN PERSON 
3 - 5 P.M. 

214 16TH STREET 



How do you apply for this job? 

A. Write a letter. 

B. Call on the telephone. 

C. Go to 21 4 1 6th Street. 

D. Send a friend. 



Sample Reading Item at CASAS Level E, 246+ 

Competency 4.3.2: Interpret work safety manuals and related information. 



Which of the following best describes 
the purpose of this information? 

A. It is a listing of common health 
hazards and possible injuries. 

B. It is a set of precautions that should 
be taken when hazardous materials 
are handled. 

C. It is a system for classifying materials 
according to the health risk they 
present. 

D. It is a table identifying the medical 
treatment indicated for various 
levels of exposure to hazardous 
materials. 



IDENTIFICATION OF HEALTH HAZARD 



Type of Possible Injury 



LEVEL 


Materials which on very short exposure 


4 


could cause death or major residual injury 
even though prompt medical treatment 




were given. 


3 


Materials which on short exposure could 
cause serious temporary or residual injury 
even though prompt medical treatment 




were given. 


Materials which on intense or continued 
exposure could cause temporary 
X . incapacitation or possible residual injury 

unless prompt medical treatment is given. 


Materials which on exposure would cause 
1 irritation but only minor residual injury even 

if no treatment is given. 


0 


Materials which on exposure under fire 
conditions would offer no hazard beyond 
that of ordinary combustibles. 



Level 3 Work Keys Reading for Information example item 



MEMO 




To: 

From: 

Re: 



All businesses in Logan City Mall 
Philip Charles, Logan City Mall Manager 
New garbage collection rules 



Logan City Mall has hired a new garbage collection company. Speedy Sanitation, Inc. will be 
collecting garbage from all businesses starting next Monday. Collection days will not change. 
The pick-up time will be one hour later. 

Each business will be given one blue garbage can to use. Each business may ask for 2 extra 
garbage cans. You may have a total of 3 garbage cans. You will not need yellow collection tags 
anymore. Full garbage cans must weigh less than 30 pounds. Put your garbage in bags before 
putting it in the garbage cans. Put your garbage cans in the alley behind your business's back 
door. 



1 . Starting next Monday, what is the greatest number of garbage cans each business is al- 
lowed to have? 



A. . 1 

B. 2 

C. 3 

D. 15 

E. 30 
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Level 7 Work Keys Reading for Information example item 



Section 108 

a) Notwithstanding the provisions of Section 1 06, it is not an infringement of copyright for 
a library or archive, or any of its employees acting within the scope of their employ- 
ment, to reproduce no more than one copy or phonorecord of a work, or to distribute 
such copy or phonorecord under the conditions specified by this section if 

1 ) the reproduction or distribution is made without any purpose of direct or indi- 
rect commercial advantage; 

2) the collections of the library or archive are (i) open to the public, or (ii) available 
not only to researchers affiliated with the library or archive or with the institution 
of which it is a part, but also to other persons doing research in a specialized 
field; and 

3) the reproduction or distribution of the work includes a notice of copyright. 

b) The rights of reproduction and distribution under this section apply to a copy or 
phonorecord of an unpublished work duplicated in facsimile form solely for purposes of 
preservation and security, or for deposit for research use in another library or archive of 
the type described by clause (2) of subsection (a), if the copy or phonorecord repro- 
duced is currently in the collections of that library or archive. 

c) The right of reproduction under this section applies to a copy or phonorecord of a 
published work duplicated in facsimile form solely for the purpose of replacement of a 
copy or phonorecord that is damaged, deteriorating, lost, or stolen, if the library or 
archive has, after a reasonable effort, determined that an unused replacement cannot be 
obtained at fair cost. 

93 



2. Based on the information above, which of the following conditions would prohibit a city 
employee from photocopying an unpublished manuscript? 

A. If the photocopy is to be sent to a public research library that does not have a copy of 
the manuscript 

B. If the photocopy would not produce any income for the city library 

C. If the city library's original copy of the manuscript is in danger of damage through use 

D. If the city library is accessible to any and all citizens and researchers 

E. If the employee makes only one copy of the manuscript as a secure transcription 
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Sample Mathematics Item at CASAS Level A, <200 



Competency 4.1 .3: Identify and use sources of information about job opportunities such as job 
descriptions, job ads, and announcements, and about the workforce arid job market. 



A B 




You started working at the time shown on clock A and stopped at the time shown on clock B. 
How long did you work? 

A. 1 hour 

B. 3 hours 

C. 5 hours 

D. 10 minutes 



Sample Mathematics Item at CASAS Level E, 246+ 

Competency 4.7.3: Identify or demonstrate effective management of human resources, 
including assessing skills, making appropriate work assignments, and 
monitoring performance. 

Wilma Santos works in a company that manufactures electrical equipment. She does the final 
assembly on regulator units. She keeps a log of the number of units she completes each day. She 
works five days a week. Below is her log for the week of August 2 through August 6. 



The total for 8/6 is how much higher than the 
total for 8/2? 

A. 4% 

B. 13.3% 

C. 25% 

D. 33% 



PRODUCTION LOG 


Name: 


Wilma Santos 


Week: 


8/2 - 8/6 




No. Units 


Date 


assembled 


8/2 


12 


8/3 


15 


8/4 


18 


8/5 


17 


8/6 


16 


Weekly total 


Daily average 
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Level 3 Work Keys Applied Mathematics 
example item 



1 . In your job as a cashier, a customer gives you a $20 bill 
to pay for a can of coffee that costs $3.84. How much 
change should you give back? 



A. 


$15.26 


B. 


$16.16 


C. 


$16.26 


D. 


$16.84 


E. 


$17.16 



Level 7 Work Keys Applied Mathematics 
example item 



2. You operate a machine that stamps bottle caps out of 3- 
inch-by-3-inch aluminum squares. Occasionally, the 
machine produces an unusable cap, a reject, that must 
be recycled. The number of rejects made at different 
production rates is shown below. Today you have been 
told to produce 600 caps per hour. Approximately how 
many caps total should you have to produce to end up 
with your quota of 2,400 good ones? 




F. 


2,400 


G. 


2,448 


H. 


2,521 


J. 


2,548 


K. 


2,616 



O 
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Appendix D 



Number of Examinees with Usable Scores by State and 
Data Collection Site 
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TabBe Do! 

Ndumbeir of Ejsamoimees wottb UsabBe Scores by Stlatte airocB Data CollllectDOin) Sate 








Mathe- 


Reading 






Reading 


matics 


and 


Total 


state 


Data Collection Site 


Only 


Only 


Mathematics 


Examinees 


‘ CALIFORNIA 


Mission College 


16 


13 


2 


31 




Mira Costa Community College 


8 


10 


5 


23 ' 




San Diego Community College 


8 


5 


7 


20 


CONNECTICUT 


Area Cooperative Educational Services 


10 


7 


8 


2 




EASTCONN 


7 


6 


5 


18 




Applied Engineering Products 


0 


0 


12 


12 




Valley Regional Adult Education 


6 


0 


0 


6 




Wallingford Adult Education 


9 


10 


4 


23 


IOWA 


North Iowa Area Community College 


12 


10 


2 


24 




Eastern Iowa Community College District 


6 


6 


3 


15 




Kirkwood Community College 


12 


5 


1 


18 




Des Moines Area Community College 


10 


13 


0 


23 




Indian Hills Community College 


5 


5 


0 


10 


KANSAS 


Fort Scott Community College 


0 


0 


13 


13 




Wichita Area Technical College 


6 


9 


5 


20 




Johnson County Community College 


2 


0 


10 


12 


MARYLAND 


USWA Bethlehem Steel Center 


8 


7 


8 


23 


MINNESOTA 


Metro East Adult Basic Education 


8 


10 


2 


20 




Carver-Scott Cooperative 


0 


0 


2 


2 




Alternative Learning Center 


5 


3 


0 


8 




Community Education Building 


7 


7 


5 


19 


NEBRASKA 


Omaha Public Schools 


10 


8 


5 


23 


OREGON 


Lane Community College 


10 


4 


6 


20 




Oregon State Corrections 


10 


9 


5 


24 




Blue Mountain Community College 


0 


1 


17 


18 




Rogue Community College 


11 


9 


5 


25 




Oregon Coast Community College 


7 


6 


6 


19 


TOTAL 




193 


163 


138 


494 



CASAS and ACT, 1997 



Appendix E 



Relationship Between Study Participants and Typical 
Examinees for Each Assessment Program 
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EelaiitioBsIhiip beUweemi Stady PasticipaiBits aiBd Typical 
lExamimiees foff Eadh Assessmeiail FirogiirainriL 



A number of the study participants were asked to complete both the reading and the mathemat- 
ics assessments so that CASAS and ACT could check the internal consistency of their assessment 
instruments. Because examinees did not represent a national random sample, it was necessary 
for each organization to determine whether the relationship between study participants' reading 
and mathematics scores closely matched that of the much wider universe of individuals tested 
by its particular assessment system. 

The correlation between the Workforce Learning Systems ECS 130 reading and mathematics 
scores for study participants was .62 (N=1 92). The comparable correlation for scores of a much 
larger group of individuals assessed by the ECS 1 30 was .65 (N=776). The correlation between 
Work Keys Reading for Information and Applied Mathematics scores for study participants was 
.60 based on the level scores (N=1 40). The comparable correlation for data from the Work Keys 
database was .58 (see Table E.1). 



Table E.l 

Conrelatiomi Coeffodemits bettweeini DmidUvodluals' 

Keadloinig amidl Matbematocs Scores for Eadhi Assessmemit System 

Population Tested Workforce Learning Systems 

ECS 130 Work Keys 



Study participants 


.62 


.60 


Larger group of individuals tested 






with the assessment instrument 


.65 


.58 



CASAS and ACT, 1997 

The similar coefficients for the two Workforce Learning Systems populations, and the similar 
coefficients for the two Work Keys populations suggest that the study participants responded to 
the two assessment systems in much the same way that other examinees had. This increases the 
likelihood that the relationships found between the Workforce Learning Systems and Work Keys 
system in this study will be applicable to a larger population as well. 
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